Jul 06 19:48:19.437160 ip-10-0-130-172 systemd[1]: Starting Kubernetes Kubelet... Jul 06 19:48:19.883986 ip-10-0-130-172 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:48:19.883986 ip-10-0-130-172 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 19:48:19.883986 ip-10-0-130-172 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:48:19.883986 ip-10-0-130-172 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 19:48:19.883986 ip-10-0-130-172 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:48:19.883986 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.869660 2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 19:48:19.883986 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874657 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:48:19.883986 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874669 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:48:19.883986 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874673 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:48:19.883986 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874677 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:48:19.883986 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874680 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:48:19.883986 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874683 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:48:19.883986 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874686 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:48:19.883986 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874689 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:48:19.900097 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874692 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:48:19.900097 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874695 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:48:19.900097 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874699 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:48:19.900097 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874702 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:48:19.900097 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874705 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:48:19.900097 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874709 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:48:19.900097 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874718 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:48:19.900097 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874721 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:48:19.900097 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874724 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:48:19.900097 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874726 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:48:19.900097 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874729 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:48:19.900097 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874731 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:48:19.900097 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874734 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:48:19.900097 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874737 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:48:19.900097 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874740 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:48:19.900097 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874742 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:48:19.900097 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874745 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:48:19.900097 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874747 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:48:19.900097 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874750 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:48:19.913540 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874753 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:48:19.913540 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874755 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:48:19.913540 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874758 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:48:19.913540 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874761 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:48:19.913540 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874765 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:48:19.913540 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874768 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:48:19.913540 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874771 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:48:19.913540 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874774 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:48:19.913540 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874776 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:48:19.913540 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874779 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:48:19.913540 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874782 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:48:19.913540 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874785 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:48:19.913540 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874788 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:48:19.913540 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874790 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:48:19.913540 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874793 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:48:19.913540 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874796 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:48:19.913540 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874816 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:48:19.913540 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874820 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:48:19.913540 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874822 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:48:20.155639 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874826 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:48:20.155639 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874833 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:48:20.155639 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874836 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:48:20.155639 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874841 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:48:20.155639 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874845 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:48:20.155639 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874848 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:48:20.155639 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874851 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:48:20.155639 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874854 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:48:20.155639 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874857 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:48:20.155639 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874860 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:48:20.155639 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874863 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:48:20.155639 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874865 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:48:20.155639 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874868 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:48:20.155639 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874871 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:48:20.155639 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874873 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:48:20.155639 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874876 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:48:20.155639 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874879 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:48:20.155639 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874881 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:48:20.155639 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874884 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:48:20.155639 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874887 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:48:19.951667 ip-10-0-130-172 systemd[1]: Started Kubernetes Kubelet. Jul 06 19:48:20.172716 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874889 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:48:20.172716 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874892 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:48:20.172716 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874895 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:48:20.172716 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874897 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:48:20.172716 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874900 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:48:20.172716 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874902 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:48:20.172716 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874906 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:48:20.172716 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874909 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:48:20.172716 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874911 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:48:20.172716 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874914 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:48:20.172716 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874916 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:48:20.172716 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874919 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:48:20.172716 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874922 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:48:20.172716 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874925 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:48:20.172716 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874929 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:48:20.172716 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874931 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:48:20.172716 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874934 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:48:20.172716 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874936 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:48:20.172716 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874941 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:48:20.172716 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874943 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:48:20.174386 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874946 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:48:20.174386 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874948 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:48:20.174386 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874951 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:48:20.174386 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874953 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:48:20.174386 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874956 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:48:20.174386 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874958 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:48:20.174386 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874961 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:48:20.174386 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874963 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:48:20.174386 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874966 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:48:20.174386 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874968 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:48:20.174386 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874971 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:48:20.174386 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874973 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:48:20.174386 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.874976 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:48:20.174386 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875693 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:48:20.174386 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875699 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:48:20.174386 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875704 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:48:20.174386 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875708 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:48:20.174386 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875712 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:48:20.174386 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875715 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:48:20.175488 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875718 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:48:20.175488 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875721 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:48:20.175488 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875723 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:48:20.175488 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875726 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:48:20.175488 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875728 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:48:20.175488 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875731 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:48:20.175488 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875733 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:48:20.175488 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875735 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:48:20.175488 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875739 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:48:20.175488 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875742 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:48:20.175488 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875744 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:48:20.175488 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875750 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:48:20.175488 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875753 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:48:20.175488 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875756 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:48:20.175488 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875759 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:48:20.175488 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875762 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:48:20.175488 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875765 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:48:20.175488 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875768 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:48:20.175488 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875770 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:48:20.175488 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875773 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:48:20.177863 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875776 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:48:20.177863 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875779 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:48:20.177863 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875782 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:48:20.177863 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875784 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:48:20.177863 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875787 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:48:20.177863 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875790 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:48:20.177863 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875793 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:48:20.177863 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875796 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:48:20.177863 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875818 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:48:20.177863 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875823 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:48:20.177863 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875826 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:48:20.177863 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875829 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:48:20.177863 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875831 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:48:20.177863 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875834 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:48:20.177863 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875836 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:48:20.177863 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875839 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:48:20.177863 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875841 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:48:20.177863 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875844 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:48:20.177863 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875846 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:48:20.177863 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875849 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:48:20.179165 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875851 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:48:20.179165 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875854 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:48:20.179165 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875857 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:48:20.179165 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875861 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:48:20.179165 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875863 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:48:20.179165 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875867 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:48:20.179165 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875869 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:48:20.179165 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875872 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:48:20.179165 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875874 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:48:20.179165 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875877 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:48:20.179165 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875881 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:48:20.179165 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875885 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:48:20.179165 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875888 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:48:20.179165 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875890 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:48:20.179165 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875893 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:48:20.179165 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875896 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:48:20.179165 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875898 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:48:20.179165 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875900 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:48:20.179165 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875903 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:48:20.180902 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875905 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:48:20.180902 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875908 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:48:20.180902 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875910 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:48:20.180902 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875913 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:48:20.180902 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875918 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:48:20.180902 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875920 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:48:20.180902 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875922 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:48:20.180902 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875925 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:48:20.180902 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875928 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:48:20.180902 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875930 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:48:20.180902 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875933 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:48:20.180902 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875935 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:48:20.180902 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875938 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:48:20.180902 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875941 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:48:20.180902 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875943 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:48:20.180902 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875946 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:48:20.180902 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875950 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:48:20.180902 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875953 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:48:20.180902 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875956 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:48:20.180902 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875958 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:48:20.182174 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875961 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:48:20.182174 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875963 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:48:20.182174 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875966 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:48:20.182174 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875968 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:48:20.182174 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875971 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:48:20.182174 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875973 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:48:20.182174 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875975 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:48:20.182174 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875978 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:48:20.182174 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875980 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:48:20.182174 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875983 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:48:20.182174 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875985 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:48:20.182174 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875988 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:48:20.182174 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875990 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:48:20.182174 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.875993 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:48:20.182174 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876359 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:48:20.182174 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876364 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:48:20.182174 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876368 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:48:20.182174 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876371 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:48:20.182174 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876373 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:48:20.182174 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876376 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:48:20.184289 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876379 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:48:20.184289 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876381 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:48:20.184289 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876384 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:48:20.184289 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876386 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:48:20.184289 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876389 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:48:20.184289 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876392 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:48:20.184289 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876394 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:48:20.184289 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876399 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:48:20.184289 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876403 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:48:20.184289 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876405 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:48:20.184289 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876408 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:48:20.184289 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876411 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:48:20.184289 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876413 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:48:20.184289 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876416 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:48:20.184289 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876418 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:48:20.184289 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876421 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:48:20.184289 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876424 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:48:20.184289 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876428 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:48:20.184289 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876432 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:48:20.184289 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876435 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:48:20.185837 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876438 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:48:20.185837 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876441 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:48:20.185837 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876444 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:48:20.185837 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876446 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:48:20.185837 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876449 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:48:20.185837 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876452 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:48:20.185837 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876455 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:48:20.185837 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876458 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:48:20.185837 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876464 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:48:20.185837 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876467 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:48:20.185837 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876470 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:48:20.185837 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876473 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:48:20.185837 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876476 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:48:20.185837 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876478 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:48:20.185837 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876481 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:48:20.185837 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876483 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:48:20.185837 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876486 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:48:20.185837 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876489 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:48:20.185837 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876491 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:48:20.188393 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876493 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:48:20.188393 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876498 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:48:20.188393 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876501 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:48:20.188393 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876504 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:48:20.188393 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876507 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:48:20.188393 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876509 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:48:20.188393 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876512 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:48:20.188393 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876515 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:48:20.188393 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876517 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:48:20.188393 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876520 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:48:20.188393 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876522 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:48:20.188393 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876525 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:48:20.188393 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876527 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:48:20.188393 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876529 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:48:20.188393 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876532 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:48:20.188393 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876534 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:48:20.188393 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876537 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:48:20.188393 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876539 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:48:20.188393 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876542 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:48:20.189468 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876545 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:48:20.189468 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876547 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:48:20.189468 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876551 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:48:20.189468 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876553 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:48:20.189468 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876556 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:48:20.189468 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876558 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:48:20.189468 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876561 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:48:20.189468 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876563 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:48:20.189468 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876567 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:48:20.189468 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876570 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:48:20.189468 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876574 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:48:20.189468 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876577 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:48:20.189468 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876579 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:48:20.189468 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876582 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:48:20.189468 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876585 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:48:20.189468 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876589 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:48:20.189468 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876591 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:48:20.189468 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876594 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:48:20.189468 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876596 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:48:20.192658 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876599 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:48:20.192658 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876601 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:48:20.192658 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876604 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:48:20.192658 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876606 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:48:20.192658 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876609 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:48:20.192658 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876611 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:48:20.192658 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876614 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:48:20.192658 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876616 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:48:20.192658 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876619 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:48:20.192658 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876621 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:48:20.192658 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876624 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:48:20.192658 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876626 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:48:20.192658 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876628 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:48:20.192658 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876631 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:48:20.192658 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876634 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:48:20.192658 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.876637 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:48:20.192658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876709 2581 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 19:48:20.192658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876717 2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 19:48:20.192658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876723 2581 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 19:48:20.192658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876728 2581 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 19:48:20.192658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876732 2581 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876735 2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876739 2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876744 2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876747 2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876750 2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876753 2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876758 2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876761 2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876764 2581 flags.go:64] FLAG: --cgroup-root="" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876767 2581 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876770 2581 flags.go:64] FLAG: --client-ca-file="" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876773 2581 flags.go:64] FLAG: --cloud-provider="external" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876776 2581 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876781 2581 flags.go:64] FLAG: --cluster-domain="" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876784 2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876787 2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876790 2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876793 2581 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876810 2581 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876816 2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876828 2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876832 2581 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876834 2581 flags.go:64] FLAG: --contention-profiling="false" Jul 06 19:48:20.193771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876837 2581 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876840 2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876843 2581 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876848 2581 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876852 2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876855 2581 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876858 2581 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876861 2581 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876864 2581 flags.go:64] FLAG: --enable-server="true" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876867 2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876871 2581 flags.go:64] FLAG: --event-burst="100" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876874 2581 flags.go:64] FLAG: --event-qps="50" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876877 2581 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876880 2581 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876883 2581 flags.go:64] FLAG: --eviction-hard="" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876887 2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876890 2581 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876893 2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876897 2581 flags.go:64] FLAG: --eviction-soft="" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876900 2581 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876903 2581 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876906 2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876908 2581 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876911 2581 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876914 2581 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 19:48:20.196886 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876917 2581 flags.go:64] FLAG: --feature-gates="" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876920 2581 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876924 2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876927 2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876930 2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876933 2581 flags.go:64] FLAG: --healthz-port="10248" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876936 2581 flags.go:64] FLAG: --help="false" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876939 2581 flags.go:64] FLAG: --hostname-override="ip-10-0-130-172.ec2.internal" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876942 2581 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876945 2581 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876949 2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876952 2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876956 2581 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876959 2581 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876962 2581 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876964 2581 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876967 2581 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876970 2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876973 2581 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876976 2581 flags.go:64] FLAG: --kube-reserved="" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876979 2581 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876981 2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876985 2581 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876988 2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 19:48:20.198128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876991 2581 flags.go:64] FLAG: --lock-file="" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876994 2581 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.876997 2581 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877000 2581 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877005 2581 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877008 2581 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877011 2581 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877014 2581 flags.go:64] FLAG: --logging-format="text" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877017 2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877020 2581 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877023 2581 flags.go:64] FLAG: --manifest-url="" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877025 2581 flags.go:64] FLAG: --manifest-url-header="" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877029 2581 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877032 2581 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877036 2581 flags.go:64] FLAG: --max-pods="110" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877039 2581 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877042 2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877045 2581 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877049 2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877052 2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877055 2581 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877058 2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877065 2581 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877068 2581 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877071 2581 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 19:48:20.200698 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877074 2581 flags.go:64] FLAG: --pod-cidr="" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877077 2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877083 2581 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877086 2581 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877089 2581 flags.go:64] FLAG: --pods-per-core="0" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877091 2581 flags.go:64] FLAG: --port="10250" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877096 2581 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877099 2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e185b6e6cd264ce9" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877102 2581 flags.go:64] FLAG: --qos-reserved="" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877105 2581 flags.go:64] FLAG: --read-only-port="10255" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877109 2581 flags.go:64] FLAG: --register-node="true" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877112 2581 flags.go:64] FLAG: --register-with-taints="" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877115 2581 flags.go:64] FLAG: --registry-burst="10" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877118 2581 flags.go:64] FLAG: --registry-qps="5" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877121 2581 flags.go:64] FLAG: --reserved-cpus="" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877123 2581 flags.go:64] FLAG: --reserved-memory="" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877127 2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877130 2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877133 2581 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877135 2581 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877138 2581 flags.go:64] FLAG: --runonce="false" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877141 2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877144 2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877147 2581 flags.go:64] FLAG: --seccomp-default="false" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877150 2581 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 19:48:20.201830 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877155 2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877158 2581 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877161 2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877164 2581 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877167 2581 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877170 2581 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877172 2581 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877175 2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877178 2581 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877181 2581 flags.go:64] FLAG: --system-cgroups="" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877183 2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877189 2581 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877191 2581 flags.go:64] FLAG: --tls-cert-file="" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877196 2581 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877201 2581 flags.go:64] FLAG: --tls-min-version="" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877204 2581 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877207 2581 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877210 2581 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877213 2581 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877216 2581 flags.go:64] FLAG: --v="2" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877220 2581 flags.go:64] FLAG: --version="false" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877224 2581 flags.go:64] FLAG: --vmodule="" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877228 2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877231 2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877350 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:48:20.204348 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877353 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:48:20.205479 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877357 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:48:20.205479 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877360 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:48:20.205479 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877362 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:48:20.205479 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877365 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:48:20.205479 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877368 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:48:20.205479 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877370 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:48:20.205479 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877375 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:48:20.205479 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877377 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:48:20.205479 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877380 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:48:20.205479 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877383 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:48:20.205479 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877385 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:48:20.205479 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877388 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:48:20.205479 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877390 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:48:20.205479 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877393 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:48:20.205479 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877395 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:48:20.205479 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877398 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:48:20.205479 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877401 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:48:20.205479 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877403 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:48:20.205479 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877405 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:48:20.205479 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877411 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:48:20.208432 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877413 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:48:20.208432 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877416 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:48:20.208432 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877419 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:48:20.208432 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877421 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:48:20.208432 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877424 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:48:20.208432 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877427 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:48:20.208432 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877429 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:48:20.208432 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877432 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:48:20.208432 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877435 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:48:20.208432 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877437 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:48:20.208432 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877442 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:48:20.208432 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877445 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:48:20.208432 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877448 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:48:20.208432 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877451 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:48:20.208432 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877454 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:48:20.208432 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877457 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:48:20.208432 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877460 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:48:20.208432 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877462 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:48:20.208432 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877467 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:48:20.209324 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877470 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:48:20.209324 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877473 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:48:20.209324 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877475 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:48:20.209324 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877478 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:48:20.209324 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877481 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:48:20.209324 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877483 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:48:20.209324 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877485 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:48:20.209324 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877488 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:48:20.209324 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877491 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:48:20.209324 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877493 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:48:20.209324 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877496 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:48:20.209324 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877498 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:48:20.209324 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877502 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:48:20.209324 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877505 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:48:20.209324 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877507 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:48:20.209324 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877510 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:48:20.209324 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877513 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:48:20.209324 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877515 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:48:20.209324 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877518 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:48:20.209324 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877520 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:48:20.212285 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877523 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:48:20.212285 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877526 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:48:20.212285 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877528 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:48:20.212285 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877531 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:48:20.212285 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877534 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:48:20.212285 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877536 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:48:20.212285 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877539 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:48:20.212285 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877541 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:48:20.212285 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877544 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:48:20.212285 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877546 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:48:20.212285 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877549 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:48:20.212285 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877553 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:48:20.212285 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877555 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:48:20.212285 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877558 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:48:20.212285 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877561 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:48:20.212285 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877563 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:48:20.212285 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877566 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:48:20.212285 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877568 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:48:20.212285 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877571 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:48:20.212285 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877573 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:48:20.213384 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877576 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:48:20.213384 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877578 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:48:20.213384 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877581 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:48:20.213384 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877584 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:48:20.213384 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877589 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:48:20.213384 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877591 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:48:20.213384 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877594 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:48:20.213384 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877596 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:48:20.213384 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877599 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:48:20.213384 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877602 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:48:20.213384 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877604 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:48:20.213384 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877606 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:48:20.213384 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877610 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:48:20.213384 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877612 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:48:20.213384 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877615 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:48:20.213384 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877618 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:48:20.213384 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877622 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:48:20.213384 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.877625 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:48:20.216279 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.877631 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:48:20.216279 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.886431 2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 19:48:20.216279 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.886447 2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 19:48:20.216279 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.887278 2581 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 19:48:20.216279 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.887286 2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 19:48:20.216279 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887345 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:48:20.216279 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887350 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:48:20.216279 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887353 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:48:20.216279 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887356 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:48:20.216279 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887360 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:48:20.216279 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887363 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:48:20.216279 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887365 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:48:20.216279 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887369 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:48:20.216279 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887372 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:48:20.216279 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887374 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:48:20.216981 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887377 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:48:20.216981 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887380 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:48:20.216981 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887382 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:48:20.216981 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887385 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:48:20.216981 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887388 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:48:20.216981 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887391 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:48:20.216981 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887394 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:48:20.216981 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887396 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:48:20.216981 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887399 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:48:20.216981 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887403 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:48:20.216981 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887407 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:48:20.216981 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887410 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:48:20.216981 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887414 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:48:20.216981 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887417 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:48:20.216981 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887420 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:48:20.216981 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887422 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:48:20.216981 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887425 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:48:20.216981 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887428 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:48:20.216981 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887430 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:48:20.434406 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887434 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:48:20.434406 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887437 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:48:20.434406 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887440 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:48:20.434406 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887442 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:48:20.434406 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887445 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:48:20.434406 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887448 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:48:20.434406 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887450 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:48:20.434406 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887453 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:48:20.434406 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887455 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:48:20.434406 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887458 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:48:20.434406 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887461 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:48:20.434406 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887463 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:48:20.434406 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887467 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:48:20.434406 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887471 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:48:20.434406 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887474 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:48:20.434406 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887477 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:48:20.434406 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887481 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:48:20.434406 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887485 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:48:20.434406 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887488 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:48:20.435453 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887491 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:48:20.435453 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887494 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:48:20.435453 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887497 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:48:20.435453 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887499 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:48:20.435453 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887502 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:48:20.435453 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887505 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:48:20.435453 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887507 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:48:20.435453 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887510 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:48:20.435453 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887513 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:48:20.435453 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887515 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:48:20.435453 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887518 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:48:20.435453 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887520 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:48:20.435453 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887523 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:48:20.435453 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887526 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:48:20.435453 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887529 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:48:20.435453 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887532 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:48:20.435453 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887534 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:48:20.435453 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887537 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:48:20.435453 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887540 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:48:20.435453 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887542 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:48:20.438389 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887545 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:48:20.438389 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887548 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:48:20.438389 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887550 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:48:20.438389 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887556 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:48:20.438389 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887558 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:48:20.438389 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887561 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:48:20.438389 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887563 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:48:20.438389 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887566 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:48:20.438389 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887569 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:48:20.438389 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887572 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:48:20.438389 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887574 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:48:20.438389 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887577 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:48:20.438389 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887580 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:48:20.438389 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887583 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:48:20.438389 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887585 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:48:20.438389 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887588 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:48:20.438389 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887591 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:48:20.438389 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887593 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:48:20.438389 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887596 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:48:20.438389 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887599 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:48:20.439280 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887602 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:48:20.439280 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887605 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:48:20.439280 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887607 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:48:20.439280 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887610 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:48:20.439280 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887612 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:48:20.439280 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887615 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:48:20.439280 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887619 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:48:20.439280 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887622 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:48:20.439280 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887624 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:48:20.439280 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887627 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:48:20.439280 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887629 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:48:20.439280 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.887635 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:48:20.439280 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887747 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:48:20.439280 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887751 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:48:20.439280 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887754 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:48:20.442464 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887759 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:48:20.442464 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887762 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:48:20.442464 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887765 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:48:20.442464 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887768 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:48:20.442464 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887770 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:48:20.442464 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887773 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:48:20.442464 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887775 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:48:20.442464 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887778 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:48:20.442464 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887781 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:48:20.442464 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887784 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:48:20.442464 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887786 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:48:20.442464 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887789 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:48:20.442464 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887791 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:48:20.442464 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887794 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:48:20.442464 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887796 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:48:20.442464 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887813 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:48:20.442464 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887816 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:48:20.442464 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887818 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:48:20.442464 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887821 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:48:20.442464 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887825 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:48:20.443373 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887830 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:48:20.443373 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887833 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:48:20.443373 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887837 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:48:20.443373 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887840 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:48:20.443373 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887844 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:48:20.443373 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887847 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:48:20.443373 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887850 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:48:20.443373 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887853 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:48:20.443373 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887855 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:48:20.443373 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887858 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:48:20.443373 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887861 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:48:20.443373 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887863 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:48:20.443373 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887867 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:48:20.443373 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887870 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:48:20.443373 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887873 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:48:20.443373 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887875 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:48:20.443373 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887878 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:48:20.443373 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887881 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:48:20.443373 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887884 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:48:20.444270 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887886 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:48:20.444270 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887889 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:48:20.444270 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887891 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:48:20.444270 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887894 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:48:20.444270 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887897 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:48:20.444270 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887900 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:48:20.444270 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887902 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:48:20.444270 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887905 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:48:20.444270 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887907 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:48:20.444270 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887910 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:48:20.444270 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887912 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:48:20.444270 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887915 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:48:20.444270 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887917 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:48:20.444270 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887920 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:48:20.444270 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887922 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:48:20.444270 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887925 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:48:20.444270 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887927 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:48:20.444270 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887931 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:48:20.444270 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887934 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:48:20.444270 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887937 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:48:20.447238 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887939 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:48:20.447238 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887942 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:48:20.447238 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887945 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:48:20.447238 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887947 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:48:20.447238 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887950 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:48:20.447238 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887954 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:48:20.447238 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887957 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:48:20.447238 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887959 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:48:20.447238 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887962 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:48:20.447238 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887965 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:48:20.447238 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887967 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:48:20.447238 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887970 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:48:20.447238 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887972 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:48:20.447238 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887975 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:48:20.447238 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887977 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:48:20.447238 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887980 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:48:20.447238 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887983 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:48:20.447238 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887986 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:48:20.447238 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887988 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:48:20.447238 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887991 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:48:20.448116 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887993 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:48:20.448116 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887996 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:48:20.448116 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.887998 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:48:20.448116 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.888001 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:48:20.448116 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.888003 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:48:20.448116 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.888006 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:48:20.448116 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.888008 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:48:20.448116 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.888011 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:48:20.448116 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.888013 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:48:20.448116 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.888017 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:48:20.448116 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.888021 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:48:20.448116 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.888024 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:48:20.448116 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.888027 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:48:20.448116 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.888030 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:48:20.448116 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.888032 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:48:20.448116 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.888035 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:48:20.448116 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:19.888037 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:48:20.449217 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.888043 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:48:20.449217 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.888151 2581 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 19:48:20.449217 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.892385 2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 19:48:20.449217 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.894040 2581 server.go:1019] "Starting client certificate rotation" Jul 06 19:48:20.449217 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.894144 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 19:48:20.449217 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.894179 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 19:48:20.449217 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.897060 2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 19:48:20.449217 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.900463 2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 19:48:20.449217 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.914204 2581 log.go:25] "Validated CRI v1 runtime API" Jul 06 19:48:20.449217 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.916902 2581 log.go:25] "Validated CRI v1 image API" Jul 06 19:48:20.449217 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.918539 2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 19:48:20.449217 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.921208 2581 fs.go:135] Filesystem UUIDs: map[2880f5f7-463b-4430-8b60-ee62b8366b44:/dev/nvme0n1p4 51eaae0e-8726-40d2-8a82-177ab5d3df60:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 06 19:48:20.451165 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.921225 2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 19:48:20.451165 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.925071 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 19:48:20.453048 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.928517 2581 manager.go:216] Machine: {Timestamp:2026-07-06 19:48:19.927161955 +0000 UTC m=+0.377917527 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096390 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2028f1b28ebe4fd12779c6fb1f920d SystemUUID:ec2028f1-b28e-be4f-d127-79c6fb1f920d BootID:b4949e22-8cab-4c1b-b149-ee12f724c5b8 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:00:52:e1:20:15 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:00:52:e1:20:15 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:66:2a:6a:e6:cc:7c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 19:48:20.453048 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.928617 2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 19:48:20.453048 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.928737 2581 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 19:48:20.453048 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.932226 2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 19:48:20.453048 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.932250 2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-172.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 19:48:20.453048 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.933103 2581 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 19:48:20.453048 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.933114 2581 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 19:48:20.453048 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.933122 2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 19:48:20.453048 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.933136 2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 19:48:20.453048 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.935012 2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 19:48:20.453048 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.939771 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 06 19:48:20.453048 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.939912 2581 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 19:48:20.453048 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.939943 2581 kubelet.go:486] "Attempting to sync node with API server" Jul 06 19:48:20.453048 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.939952 2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 19:48:20.453048 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.939966 2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 19:48:20.453048 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.939976 2581 kubelet.go:398] "Adding apiserver pod source" Jul 06 19:48:20.453048 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.939985 2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 19:48:20.453048 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.940316 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.940332 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.942834 2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.945117 2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.945143 2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.946504 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.946526 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.946536 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.946545 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.946559 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.946569 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.946578 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.946582 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.946590 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.946595 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.946604 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.946609 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.947496 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.947505 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.950719 2581 server.go:1268] "Started kubelet" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.950812 2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.950813 2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.950901 2581 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.952240 2581 server.go:310] "Adding debug handlers to kubelet server" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.952925 2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.956033 2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-172.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:19.956347 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:19.956385 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-172.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:19.957102 2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:19.956031 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-172.ec2.internal.18bfcb1c519bc0d9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-172.ec2.internal,UID:ip-10-0-130-172.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-172.ec2.internal,},FirstTimestamp:2026-07-06 19:48:19.950690521 +0000 UTC m=+0.401446094,LastTimestamp:2026-07-06 19:48:19.950690521 +0000 UTC m=+0.401446094,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-172.ec2.internal,}" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.961699 2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.961710 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:19.961771 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-172.ec2.internal\" not found" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.962413 2581 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.962416 2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.962448 2581 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.962564 2581 reconstruct.go:97] "Volume reconstruction finished" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.962571 2581 reconciler.go:29] "Reconciler: start to sync state" Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.964148 2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.964159 2581 factory.go:55] Registering systemd factory Jul 06 19:48:20.454142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.964165 2581 factory.go:223] Registration of the systemd container factory successfully Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.964353 2581 factory.go:153] Registering CRI-O factory Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.964369 2581 factory.go:223] Registration of the crio container factory successfully Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.964396 2581 factory.go:103] Registering Raw factory Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.964410 2581 manager.go:1196] Started watching for new ooms in manager Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.965302 2581 manager.go:349] Starting recovery of all containers Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.971846 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gpg76" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:19.973286 2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-172.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:19.973407 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.978349 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gpg76" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.979344 2581 manager.go:354] Recovery completed Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.983816 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.986941 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-172.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.986971 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-172.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.986984 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-172.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.987467 2581 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.987476 2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.987490 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.987491 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.988338 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-172.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.988366 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-172.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.988380 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-172.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:19.989134 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-172.ec2.internal.18bfcb1c53c51b6f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-172.ec2.internal,UID:ip-10-0-130-172.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-172.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-172.ec2.internal,},FirstTimestamp:2026-07-06 19:48:19.986955119 +0000 UTC m=+0.437710695,LastTimestamp:2026-07-06 19:48:19.986955119 +0000 UTC m=+0.437710695,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-172.ec2.internal,}" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.989502 2581 policy_none.go:49] "None policy: Start" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.989516 2581 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.989526 2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:19.991461 2581 policy_none.go:47] "Start" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.039420 2581 manager.go:337] "Starting Device Plugin manager" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:20.039612 2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.039623 2581 server.go:87] "Starting device plugin registration server" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.039843 2581 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.039852 2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.039961 2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.040038 2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.040047 2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:20.040538 2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:20.040593 2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-172.ec2.internal\" not found" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.083582 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.084782 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.084848 2581 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.084876 2581 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 19:48:20.660300 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:20.084919 2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.087561 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.140165 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.141036 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-172.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.141070 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-172.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.141081 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-172.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.141105 2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-172.ec2.internal" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.150575 2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-172.ec2.internal" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:20.150599 2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-172.ec2.internal\": node \"ip-10-0-130-172.ec2.internal\" not found" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:20.175217 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-172.ec2.internal\" not found" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.185016 2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-172.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-172.ec2.internal"] Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.185075 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.186496 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-172.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.186516 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-172.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.186525 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-172.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.188449 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.188593 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-172.ec2.internal" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.188619 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.189162 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-172.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.189179 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-172.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.189189 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-172.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.189379 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-172.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.189390 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-172.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.189399 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-172.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.192013 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-172.ec2.internal" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.192040 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.192740 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-172.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.192756 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-172.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.192765 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-172.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:20.251586 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-172.ec2.internal\" not found" node="ip-10-0-130-172.ec2.internal" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:20.256523 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-172.ec2.internal\" not found" node="ip-10-0-130-172.ec2.internal" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.264885 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8478f697752cbff42288357201271ac7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-172.ec2.internal\" (UID: \"8478f697752cbff42288357201271ac7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-172.ec2.internal" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.264920 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8478f697752cbff42288357201271ac7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-172.ec2.internal\" (UID: \"8478f697752cbff42288357201271ac7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-172.ec2.internal" Jul 06 19:48:20.662216 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.264938 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8812863089e77e28db706ad567f2f4d4-config\") pod \"kube-apiserver-proxy-ip-10-0-130-172.ec2.internal\" (UID: \"8812863089e77e28db706ad567f2f4d4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-172.ec2.internal" Jul 06 19:48:21.852017 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:20.275632 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-172.ec2.internal\" not found" Jul 06 19:48:21.852017 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.365173 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8478f697752cbff42288357201271ac7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-172.ec2.internal\" (UID: \"8478f697752cbff42288357201271ac7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-172.ec2.internal" Jul 06 19:48:21.852017 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.365206 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8478f697752cbff42288357201271ac7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-172.ec2.internal\" (UID: \"8478f697752cbff42288357201271ac7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-172.ec2.internal" Jul 06 19:48:21.852017 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.365222 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8812863089e77e28db706ad567f2f4d4-config\") pod \"kube-apiserver-proxy-ip-10-0-130-172.ec2.internal\" (UID: \"8812863089e77e28db706ad567f2f4d4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-172.ec2.internal" Jul 06 19:48:21.852017 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.365268 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8812863089e77e28db706ad567f2f4d4-config\") pod \"kube-apiserver-proxy-ip-10-0-130-172.ec2.internal\" (UID: \"8812863089e77e28db706ad567f2f4d4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-172.ec2.internal" Jul 06 19:48:21.852017 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.365290 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8478f697752cbff42288357201271ac7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-172.ec2.internal\" (UID: \"8478f697752cbff42288357201271ac7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-172.ec2.internal" Jul 06 19:48:21.852017 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.365290 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8478f697752cbff42288357201271ac7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-172.ec2.internal\" (UID: \"8478f697752cbff42288357201271ac7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-172.ec2.internal" Jul 06 19:48:21.852017 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:20.376281 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-172.ec2.internal\" not found" Jul 06 19:48:21.852017 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:20.476961 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-172.ec2.internal\" not found" Jul 06 19:48:21.852017 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.558153 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-172.ec2.internal" Jul 06 19:48:21.852017 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:20.560308 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-172.ec2.internal" Jul 06 19:48:21.852017 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:20.577906 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-172.ec2.internal\" not found" Jul 06 19:48:21.852017 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:20.678489 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-172.ec2.internal\" not found" Jul 06 19:48:21.852017 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:20.779482 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-172.ec2.internal\" not found" Jul 06 19:48:21.873817 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.873795 2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:48:21.896861 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.896827 2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 19:48:21.897473 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.896949 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 19:48:21.897473 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.896975 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 19:48:21.944143 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.944114 2581 apiserver.go:52] "Watching apiserver" Jul 06 19:48:21.966299 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.966274 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 19:48:21.966299 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.966287 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-172.ec2.internal" Jul 06 19:48:21.968932 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.968914 2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 19:48:21.969792 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.969772 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p","openshift-dns/node-resolver-gr86q","openshift-image-registry/node-ca-ldkxm","openshift-multus/network-metrics-daemon-tgl28","openshift-network-diagnostics/network-check-target-4wsff","openshift-ovn-kubernetes/ovnkube-node-sklff","kube-system/konnectivity-agent-zwmlp","openshift-cluster-node-tuning-operator/tuned-l58vr","openshift-multus/multus-additional-cni-plugins-wlcd6","openshift-multus/multus-cktbc","openshift-network-operator/iptables-alerter-cwnxx","kube-system/global-pull-secret-syncer-9qrkq"] Jul 06 19:48:21.972127 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.972110 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:21.974013 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.973987 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gr86q" Jul 06 19:48:21.974394 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.974378 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ljzt4\"" Jul 06 19:48:21.974479 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.974413 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 19:48:21.974479 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.974456 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 19:48:21.974588 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.974456 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 19:48:21.975905 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.975888 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ldkxm" Jul 06 19:48:21.977878 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.977863 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:21.977947 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:21.977916 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tgl28" podUID="2df21624-1cb1-4437-99a7-b91b4784d32d" Jul 06 19:48:21.978363 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.978347 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 19:48:21.978555 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.978541 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 19:48:21.980129 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.979939 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:21.980129 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.979992 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 19:48:21.980129 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:21.980008 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4wsff" podUID="7e42834f-b24c-4470-bf13-6351e2509148" Jul 06 19:48:21.980129 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.980018 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 19:48:21.980129 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.980020 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 19:48:21.980129 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.980051 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qphsx\"" Jul 06 19:48:21.980129 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.980079 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-g72mg\"" Jul 06 19:48:21.982161 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.982148 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:21.982578 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.982560 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 19:48:21.983884 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.983870 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 19:48:21.983950 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.983928 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-172.ec2.internal" Jul 06 19:48:21.984145 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.984135 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zwmlp" Jul 06 19:48:21.984660 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.984640 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 19:43:20 +0000 UTC" deadline="2028-03-15 10:26:32.516408428 +0000 UTC" Jul 06 19:48:21.984660 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.984659 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14822h38m10.531751958s" Jul 06 19:48:21.985199 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.985180 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 19:48:21.985521 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.985473 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 19:48:21.985673 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.985529 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 19:48:21.985673 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.985541 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 19:48:21.985878 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.985864 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5jbvp\"" Jul 06 19:48:21.986235 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.986212 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:21.986768 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.986747 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 19:48:21.986768 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.986767 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 19:48:21.986891 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.986834 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 19:48:21.987527 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.987512 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mt64b\"" Jul 06 19:48:21.987595 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.987580 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 19:48:21.988524 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.988509 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:21.989759 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.989700 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 19:48:21.990077 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.990011 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8wh47\"" Jul 06 19:48:21.990077 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.990022 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:48:21.992105 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.991292 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 19:48:21.992105 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.991527 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 19:48:21.992105 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.991533 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 19:48:21.992105 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.991730 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 19:48:21.992105 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.991842 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 19:48:21.992105 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.992059 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-m6nzx\"" Jul 06 19:48:21.992466 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.992447 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cktbc" Jul 06 19:48:21.994765 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.994751 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cwnxx" Jul 06 19:48:21.995169 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.995154 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zkwvf\"" Jul 06 19:48:21.995434 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.995422 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 19:48:21.996508 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.996492 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 19:48:21.996856 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.996841 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-172.ec2.internal"] Jul 06 19:48:21.996908 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.996866 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-172.ec2.internal"] Jul 06 19:48:21.996952 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.996915 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:21.997008 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:21.996991 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9qrkq" podUID="baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6" Jul 06 19:48:21.997245 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.997230 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 19:48:21.997508 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.997493 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 19:48:21.997565 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.997496 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xttvv\"" Jul 06 19:48:21.997714 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:21.997703 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:48:22.010049 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.010033 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vmsb6" Jul 06 19:48:22.016104 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.016087 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vmsb6" Jul 06 19:48:22.025955 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.025925 2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:48:22.067362 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.067343 2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 19:48:22.072826 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.072803 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-etc-systemd\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.072930 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.072828 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-os-release\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.072930 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.072846 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-hostroot\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.072930 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.072860 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs\") pod \"network-metrics-daemon-tgl28\" (UID: \"2df21624-1cb1-4437-99a7-b91b4784d32d\") " pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:22.072930 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.072875 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-dbus\") pod \"global-pull-secret-syncer-9qrkq\" (UID: \"baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6\") " pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:22.072930 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.072918 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret\") pod \"global-pull-secret-syncer-9qrkq\" (UID: \"baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6\") " pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:22.073091 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.072945 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b195d258-855f-47b7-a90f-9c1751cf0fe0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w8h7p\" (UID: \"b195d258-855f-47b7-a90f-9c1751cf0fe0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:22.073091 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.072963 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-host-kubelet\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.073091 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.072978 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-var-lib-kubelet\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.073091 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.072993 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qwpw\" (UniqueName: \"kubernetes.io/projected/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-kube-api-access-5qwpw\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.073091 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073008 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-system-cni-dir\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.073091 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073022 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-run-ovn\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.073091 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073048 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-host-run-ovn-kubernetes\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.073091 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073070 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.073379 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073101 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0ffad6cc-ac00-4259-92dd-648660f40808-ovn-node-metrics-cert\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.073379 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073118 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0ffad6cc-ac00-4259-92dd-648660f40808-ovnkube-script-lib\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.073379 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073133 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-etc-sysconfig\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.073379 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073147 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad86aa92-d126-4e69-a979-247c6deb986f-tmp-dir\") pod \"node-resolver-gr86q\" (UID: \"ad86aa92-d126-4e69-a979-247c6deb986f\") " pod="openshift-dns/node-resolver-gr86q" Jul 06 19:48:22.073379 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073161 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b195d258-855f-47b7-a90f-9c1751cf0fe0-etc-selinux\") pod \"aws-ebs-csi-driver-node-w8h7p\" (UID: \"b195d258-855f-47b7-a90f-9c1751cf0fe0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:22.073379 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073181 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-run-systemd\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.073379 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073206 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-etc-openvswitch\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.073379 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073228 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrf6q\" (UniqueName: \"kubernetes.io/projected/0ffad6cc-ac00-4259-92dd-648660f40808-kube-api-access-hrf6q\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.073379 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073250 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-etc-modprobe-d\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.073379 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073265 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.073379 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073280 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-system-cni-dir\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.073379 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073296 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b195d258-855f-47b7-a90f-9c1751cf0fe0-sys-fs\") pod \"aws-ebs-csi-driver-node-w8h7p\" (UID: \"b195d258-855f-47b7-a90f-9c1751cf0fe0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:22.073379 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073315 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-multus-conf-dir\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.073379 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073346 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-etc-kubernetes\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.073379 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073361 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d9cdca6a-9d67-4aae-980d-2d1800413281-iptables-alerter-script\") pod \"iptables-alerter-cwnxx\" (UID: \"d9cdca6a-9d67-4aae-980d-2d1800413281\") " pod="openshift-network-operator/iptables-alerter-cwnxx" Jul 06 19:48:22.073379 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073376 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b195d258-855f-47b7-a90f-9c1751cf0fe0-device-dir\") pod \"aws-ebs-csi-driver-node-w8h7p\" (UID: \"b195d258-855f-47b7-a90f-9c1751cf0fe0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:22.073903 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073390 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-lib-modules\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.073903 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073423 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.073903 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073444 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d9cdca6a-9d67-4aae-980d-2d1800413281-host-slash\") pod \"iptables-alerter-cwnxx\" (UID: \"d9cdca6a-9d67-4aae-980d-2d1800413281\") " pod="openshift-network-operator/iptables-alerter-cwnxx" Jul 06 19:48:22.073903 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073501 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-kubelet-config\") pod \"global-pull-secret-syncer-9qrkq\" (UID: \"baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6\") " pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:22.073903 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073530 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-etc-tuned\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.073903 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073552 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-host-run-multus-certs\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.073903 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073575 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dgrz\" (UniqueName: \"kubernetes.io/projected/ad86aa92-d126-4e69-a979-247c6deb986f-kube-api-access-5dgrz\") pod \"node-resolver-gr86q\" (UID: \"ad86aa92-d126-4e69-a979-247c6deb986f\") " pod="openshift-dns/node-resolver-gr86q" Jul 06 19:48:22.073903 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073596 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b195d258-855f-47b7-a90f-9c1751cf0fe0-registration-dir\") pod \"aws-ebs-csi-driver-node-w8h7p\" (UID: \"b195d258-855f-47b7-a90f-9c1751cf0fe0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:22.073903 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073613 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0ffad6cc-ac00-4259-92dd-648660f40808-ovnkube-config\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.073903 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073626 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0ffad6cc-ac00-4259-92dd-648660f40808-env-overrides\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.073903 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073657 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-host-var-lib-kubelet\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.073903 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073696 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-multus-daemon-config\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.073903 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073727 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjg8j\" (UniqueName: \"kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j\") pod \"network-check-target-4wsff\" (UID: \"7e42834f-b24c-4470-bf13-6351e2509148\") " pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:22.073903 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073761 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt6bz\" (UniqueName: \"kubernetes.io/projected/d9cdca6a-9d67-4aae-980d-2d1800413281-kube-api-access-vt6bz\") pod \"iptables-alerter-cwnxx\" (UID: \"d9cdca6a-9d67-4aae-980d-2d1800413281\") " pod="openshift-network-operator/iptables-alerter-cwnxx" Jul 06 19:48:22.073903 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073782 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-host-cni-netd\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.073903 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073802 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-cnibin\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.074433 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073823 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a59fb421-1d29-4686-8512-af1ee3473b27-agent-certs\") pod \"konnectivity-agent-zwmlp\" (UID: \"a59fb421-1d29-4686-8512-af1ee3473b27\") " pod="kube-system/konnectivity-agent-zwmlp" Jul 06 19:48:22.074433 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073844 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-systemd-units\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.074433 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073866 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-run-openvswitch\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.074433 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073889 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-node-log\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.074433 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073924 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-tmp\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.074433 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073954 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-multus-cni-dir\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.074433 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073975 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-host-run-netns\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.074433 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.073992 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-host-var-lib-cni-bin\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.074433 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074005 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-host-var-lib-cni-multus\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.074433 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074019 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/03659b3b-23b9-4ce3-9c69-d76c580fd1b7-serviceca\") pod \"node-ca-ldkxm\" (UID: \"03659b3b-23b9-4ce3-9c69-d76c580fd1b7\") " pod="openshift-image-registry/node-ca-ldkxm" Jul 06 19:48:22.074433 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074033 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a59fb421-1d29-4686-8512-af1ee3473b27-konnectivity-ca\") pod \"konnectivity-agent-zwmlp\" (UID: \"a59fb421-1d29-4686-8512-af1ee3473b27\") " pod="kube-system/konnectivity-agent-zwmlp" Jul 06 19:48:22.074433 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074070 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-host-run-netns\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.074433 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074098 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-cnibin\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.074433 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074122 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-os-release\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.074433 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074139 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-cni-binary-copy\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.074433 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074158 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ad86aa92-d126-4e69-a979-247c6deb986f-hosts-file\") pod \"node-resolver-gr86q\" (UID: \"ad86aa92-d126-4e69-a979-247c6deb986f\") " pod="openshift-dns/node-resolver-gr86q" Jul 06 19:48:22.074433 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074187 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckvzv\" (UniqueName: \"kubernetes.io/projected/03659b3b-23b9-4ce3-9c69-d76c580fd1b7-kube-api-access-ckvzv\") pod \"node-ca-ldkxm\" (UID: \"03659b3b-23b9-4ce3-9c69-d76c580fd1b7\") " pod="openshift-image-registry/node-ca-ldkxm" Jul 06 19:48:22.074943 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074205 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b195d258-855f-47b7-a90f-9c1751cf0fe0-socket-dir\") pod \"aws-ebs-csi-driver-node-w8h7p\" (UID: \"b195d258-855f-47b7-a90f-9c1751cf0fe0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:22.074943 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074229 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-host-slash\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.074943 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074245 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-host-cni-bin\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.074943 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074265 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-etc-kubernetes\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.074943 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074285 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-host\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.074943 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074300 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-multus-socket-dir-parent\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.074943 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074314 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03659b3b-23b9-4ce3-9c69-d76c580fd1b7-host\") pod \"node-ca-ldkxm\" (UID: \"03659b3b-23b9-4ce3-9c69-d76c580fd1b7\") " pod="openshift-image-registry/node-ca-ldkxm" Jul 06 19:48:22.074943 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074357 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-etc-sysctl-conf\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.074943 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074383 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-run\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.074943 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074407 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg49p\" (UniqueName: \"kubernetes.io/projected/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-kube-api-access-rg49p\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.074943 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074443 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc5sj\" (UniqueName: \"kubernetes.io/projected/b195d258-855f-47b7-a90f-9c1751cf0fe0-kube-api-access-xc5sj\") pod \"aws-ebs-csi-driver-node-w8h7p\" (UID: \"b195d258-855f-47b7-a90f-9c1751cf0fe0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:22.074943 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074465 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-var-lib-openvswitch\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.074943 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074486 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-etc-sysctl-d\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.074943 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074516 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-sys\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.074943 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074535 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.074943 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074550 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-host-run-k8s-cni-cncf-io\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.075500 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074564 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbwjb\" (UniqueName: \"kubernetes.io/projected/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-kube-api-access-pbwjb\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.075500 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074581 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh4bc\" (UniqueName: \"kubernetes.io/projected/2df21624-1cb1-4437-99a7-b91b4784d32d-kube-api-access-wh4bc\") pod \"network-metrics-daemon-tgl28\" (UID: \"2df21624-1cb1-4437-99a7-b91b4784d32d\") " pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:22.075500 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074600 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-log-socket\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.075500 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.074614 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-cni-binary-copy\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.175629 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.175602 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-cni-binary-copy\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.175629 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.175633 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-etc-systemd\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.175900 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.175650 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-os-release\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.175900 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.175665 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-hostroot\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.175900 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.175684 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs\") pod \"network-metrics-daemon-tgl28\" (UID: \"2df21624-1cb1-4437-99a7-b91b4784d32d\") " pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:22.175900 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.175705 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-dbus\") pod \"global-pull-secret-syncer-9qrkq\" (UID: \"baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6\") " pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:22.175900 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.175746 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-etc-systemd\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.175900 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.175751 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-hostroot\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.175900 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.175750 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-os-release\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.175900 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.175794 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret\") pod \"global-pull-secret-syncer-9qrkq\" (UID: \"baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6\") " pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:22.175900 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.175827 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b195d258-855f-47b7-a90f-9c1751cf0fe0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w8h7p\" (UID: \"b195d258-855f-47b7-a90f-9c1751cf0fe0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:22.175900 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:22.175829 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:48:22.175900 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.175860 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-host-kubelet\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.175900 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.175872 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-dbus\") pod \"global-pull-secret-syncer-9qrkq\" (UID: \"baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6\") " pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:22.175900 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.175879 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-var-lib-kubelet\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.175900 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.175881 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b195d258-855f-47b7-a90f-9c1751cf0fe0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-w8h7p\" (UID: \"b195d258-855f-47b7-a90f-9c1751cf0fe0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:22.175900 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:22.175851 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:48:22.175900 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.175912 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qwpw\" (UniqueName: \"kubernetes.io/projected/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-kube-api-access-5qwpw\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.176675 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.175914 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-host-kubelet\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.176675 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.175916 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-var-lib-kubelet\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.176675 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:22.175957 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs podName:2df21624-1cb1-4437-99a7-b91b4784d32d nodeName:}" failed. No retries permitted until 2026-07-06 19:48:22.675910666 +0000 UTC m=+2.123147387 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs") pod "network-metrics-daemon-tgl28" (UID: "2df21624-1cb1-4437-99a7-b91b4784d32d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:48:22.176675 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:22.175997 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret podName:baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:22.675986114 +0000 UTC m=+2.123222813 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret") pod "global-pull-secret-syncer-9qrkq" (UID: "baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:48:22.176675 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176015 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-system-cni-dir\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.176675 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176041 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-run-ovn\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.176675 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176066 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-host-run-ovn-kubernetes\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.176675 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176078 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-system-cni-dir\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.176675 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176092 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.176675 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176117 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0ffad6cc-ac00-4259-92dd-648660f40808-ovn-node-metrics-cert\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.176675 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176120 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-run-ovn\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.176675 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176126 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-host-run-ovn-kubernetes\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.176675 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176143 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0ffad6cc-ac00-4259-92dd-648660f40808-ovnkube-script-lib\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.176675 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176151 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-cni-binary-copy\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.176675 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176179 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.177416 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176287 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-etc-sysconfig\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.177416 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176330 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad86aa92-d126-4e69-a979-247c6deb986f-tmp-dir\") pod \"node-resolver-gr86q\" (UID: \"ad86aa92-d126-4e69-a979-247c6deb986f\") " pod="openshift-dns/node-resolver-gr86q" Jul 06 19:48:22.177416 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176363 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b195d258-855f-47b7-a90f-9c1751cf0fe0-etc-selinux\") pod \"aws-ebs-csi-driver-node-w8h7p\" (UID: \"b195d258-855f-47b7-a90f-9c1751cf0fe0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:22.177416 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176370 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-etc-sysconfig\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.177416 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176386 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-run-systemd\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.177416 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176407 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-etc-openvswitch\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.177416 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176430 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrf6q\" (UniqueName: \"kubernetes.io/projected/0ffad6cc-ac00-4259-92dd-648660f40808-kube-api-access-hrf6q\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.177416 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176453 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-etc-modprobe-d\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.177416 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176464 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-run-systemd\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.177416 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176476 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.177416 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176500 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-system-cni-dir\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.177416 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176503 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b195d258-855f-47b7-a90f-9c1751cf0fe0-etc-selinux\") pod \"aws-ebs-csi-driver-node-w8h7p\" (UID: \"b195d258-855f-47b7-a90f-9c1751cf0fe0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:22.177416 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176523 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b195d258-855f-47b7-a90f-9c1751cf0fe0-sys-fs\") pod \"aws-ebs-csi-driver-node-w8h7p\" (UID: \"b195d258-855f-47b7-a90f-9c1751cf0fe0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:22.177416 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176550 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-multus-conf-dir\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.177416 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176572 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-etc-kubernetes\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.177416 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176561 2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 19:48:22.177416 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176599 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d9cdca6a-9d67-4aae-980d-2d1800413281-iptables-alerter-script\") pod \"iptables-alerter-cwnxx\" (UID: \"d9cdca6a-9d67-4aae-980d-2d1800413281\") " pod="openshift-network-operator/iptables-alerter-cwnxx" Jul 06 19:48:22.178214 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176622 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b195d258-855f-47b7-a90f-9c1751cf0fe0-device-dir\") pod \"aws-ebs-csi-driver-node-w8h7p\" (UID: \"b195d258-855f-47b7-a90f-9c1751cf0fe0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:22.178214 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176640 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-lib-modules\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.178214 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176639 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.178214 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176662 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.178214 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176686 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad86aa92-d126-4e69-a979-247c6deb986f-tmp-dir\") pod \"node-resolver-gr86q\" (UID: \"ad86aa92-d126-4e69-a979-247c6deb986f\") " pod="openshift-dns/node-resolver-gr86q" Jul 06 19:48:22.178214 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176691 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-etc-openvswitch\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.178214 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176686 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d9cdca6a-9d67-4aae-980d-2d1800413281-host-slash\") pod \"iptables-alerter-cwnxx\" (UID: \"d9cdca6a-9d67-4aae-980d-2d1800413281\") " pod="openshift-network-operator/iptables-alerter-cwnxx" Jul 06 19:48:22.178214 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176692 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-etc-kubernetes\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.178214 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176725 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-multus-conf-dir\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.178214 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176725 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-kubelet-config\") pod \"global-pull-secret-syncer-9qrkq\" (UID: \"baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6\") " pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:22.178214 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176752 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-etc-tuned\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.178214 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176771 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-kubelet-config\") pod \"global-pull-secret-syncer-9qrkq\" (UID: \"baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6\") " pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:22.178214 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176782 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0ffad6cc-ac00-4259-92dd-648660f40808-ovnkube-script-lib\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.178214 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176794 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b195d258-855f-47b7-a90f-9c1751cf0fe0-sys-fs\") pod \"aws-ebs-csi-driver-node-w8h7p\" (UID: \"b195d258-855f-47b7-a90f-9c1751cf0fe0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:22.178214 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176753 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-system-cni-dir\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.178214 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176788 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-etc-modprobe-d\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.178214 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176824 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d9cdca6a-9d67-4aae-980d-2d1800413281-host-slash\") pod \"iptables-alerter-cwnxx\" (UID: \"d9cdca6a-9d67-4aae-980d-2d1800413281\") " pod="openshift-network-operator/iptables-alerter-cwnxx" Jul 06 19:48:22.179033 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176838 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-lib-modules\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.179033 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176849 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b195d258-855f-47b7-a90f-9c1751cf0fe0-device-dir\") pod \"aws-ebs-csi-driver-node-w8h7p\" (UID: \"b195d258-855f-47b7-a90f-9c1751cf0fe0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:22.179033 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176865 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-host-run-multus-certs\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.179033 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176893 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dgrz\" (UniqueName: \"kubernetes.io/projected/ad86aa92-d126-4e69-a979-247c6deb986f-kube-api-access-5dgrz\") pod \"node-resolver-gr86q\" (UID: \"ad86aa92-d126-4e69-a979-247c6deb986f\") " pod="openshift-dns/node-resolver-gr86q" Jul 06 19:48:22.179033 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176916 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b195d258-855f-47b7-a90f-9c1751cf0fe0-registration-dir\") pod \"aws-ebs-csi-driver-node-w8h7p\" (UID: \"b195d258-855f-47b7-a90f-9c1751cf0fe0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:22.179033 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176961 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-host-run-multus-certs\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.179033 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.176986 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0ffad6cc-ac00-4259-92dd-648660f40808-ovnkube-config\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.179033 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177002 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0ffad6cc-ac00-4259-92dd-648660f40808-env-overrides\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.179033 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177017 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-host-var-lib-kubelet\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.179033 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177032 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-multus-daemon-config\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.179033 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177046 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjg8j\" (UniqueName: \"kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j\") pod \"network-check-target-4wsff\" (UID: \"7e42834f-b24c-4470-bf13-6351e2509148\") " pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:22.179033 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177063 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt6bz\" (UniqueName: \"kubernetes.io/projected/d9cdca6a-9d67-4aae-980d-2d1800413281-kube-api-access-vt6bz\") pod \"iptables-alerter-cwnxx\" (UID: \"d9cdca6a-9d67-4aae-980d-2d1800413281\") " pod="openshift-network-operator/iptables-alerter-cwnxx" Jul 06 19:48:22.179033 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177078 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-host-cni-netd\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.179033 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177137 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-cnibin\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.179033 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177152 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-host-var-lib-kubelet\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.179033 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177161 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a59fb421-1d29-4686-8512-af1ee3473b27-agent-certs\") pod \"konnectivity-agent-zwmlp\" (UID: \"a59fb421-1d29-4686-8512-af1ee3473b27\") " pod="kube-system/konnectivity-agent-zwmlp" Jul 06 19:48:22.179033 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177176 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b195d258-855f-47b7-a90f-9c1751cf0fe0-registration-dir\") pod \"aws-ebs-csi-driver-node-w8h7p\" (UID: \"b195d258-855f-47b7-a90f-9c1751cf0fe0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:22.179840 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177341 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d9cdca6a-9d67-4aae-980d-2d1800413281-iptables-alerter-script\") pod \"iptables-alerter-cwnxx\" (UID: \"d9cdca6a-9d67-4aae-980d-2d1800413281\") " pod="openshift-network-operator/iptables-alerter-cwnxx" Jul 06 19:48:22.179840 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177539 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-host-cni-netd\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.179840 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177578 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-systemd-units\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.179840 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177606 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-run-openvswitch\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.179840 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177632 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0ffad6cc-ac00-4259-92dd-648660f40808-ovnkube-config\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.179840 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177633 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-node-log\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.179840 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177679 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-tmp\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.179840 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177685 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-systemd-units\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.179840 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177703 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-multus-cni-dir\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.179840 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177728 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-node-log\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.179840 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177728 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-host-run-netns\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.179840 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177764 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-host-run-netns\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.179840 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177798 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-run-openvswitch\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.179840 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177838 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-multus-daemon-config\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.179840 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177764 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-host-var-lib-cni-bin\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.179840 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177890 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-host-var-lib-cni-multus\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.179840 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177917 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/03659b3b-23b9-4ce3-9c69-d76c580fd1b7-serviceca\") pod \"node-ca-ldkxm\" (UID: \"03659b3b-23b9-4ce3-9c69-d76c580fd1b7\") " pod="openshift-image-registry/node-ca-ldkxm" Jul 06 19:48:22.179840 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177942 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a59fb421-1d29-4686-8512-af1ee3473b27-konnectivity-ca\") pod \"konnectivity-agent-zwmlp\" (UID: \"a59fb421-1d29-4686-8512-af1ee3473b27\") " pod="kube-system/konnectivity-agent-zwmlp" Jul 06 19:48:22.180453 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177966 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-host-run-netns\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.180453 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177989 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-cnibin\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.180453 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178013 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-os-release\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.180453 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178036 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-cni-binary-copy\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.180453 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178060 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ad86aa92-d126-4e69-a979-247c6deb986f-hosts-file\") pod \"node-resolver-gr86q\" (UID: \"ad86aa92-d126-4e69-a979-247c6deb986f\") " pod="openshift-dns/node-resolver-gr86q" Jul 06 19:48:22.180453 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178084 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckvzv\" (UniqueName: \"kubernetes.io/projected/03659b3b-23b9-4ce3-9c69-d76c580fd1b7-kube-api-access-ckvzv\") pod \"node-ca-ldkxm\" (UID: \"03659b3b-23b9-4ce3-9c69-d76c580fd1b7\") " pod="openshift-image-registry/node-ca-ldkxm" Jul 06 19:48:22.180453 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178105 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.180453 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178107 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b195d258-855f-47b7-a90f-9c1751cf0fe0-socket-dir\") pod \"aws-ebs-csi-driver-node-w8h7p\" (UID: \"b195d258-855f-47b7-a90f-9c1751cf0fe0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:22.180453 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.177596 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-cnibin\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.180453 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178158 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-host-slash\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.180453 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178181 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-host-cni-bin\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.180453 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178188 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b195d258-855f-47b7-a90f-9c1751cf0fe0-socket-dir\") pod \"aws-ebs-csi-driver-node-w8h7p\" (UID: \"b195d258-855f-47b7-a90f-9c1751cf0fe0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:22.180453 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178203 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-etc-kubernetes\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.180453 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178224 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-host\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.180453 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178252 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-multus-socket-dir-parent\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.180453 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178260 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-host-var-lib-cni-multus\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.180453 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178276 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03659b3b-23b9-4ce3-9c69-d76c580fd1b7-host\") pod \"node-ca-ldkxm\" (UID: \"03659b3b-23b9-4ce3-9c69-d76c580fd1b7\") " pod="openshift-image-registry/node-ca-ldkxm" Jul 06 19:48:22.180453 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178300 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-etc-sysctl-conf\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.181172 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178301 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0ffad6cc-ac00-4259-92dd-648660f40808-env-overrides\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.181172 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178376 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-host-slash\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.181172 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178401 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-os-release\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.181172 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178417 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-host-cni-bin\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.181172 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178438 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-run\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.181172 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178466 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-etc-kubernetes\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.181172 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178467 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg49p\" (UniqueName: \"kubernetes.io/projected/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-kube-api-access-rg49p\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.181172 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178500 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc5sj\" (UniqueName: \"kubernetes.io/projected/b195d258-855f-47b7-a90f-9c1751cf0fe0-kube-api-access-xc5sj\") pod \"aws-ebs-csi-driver-node-w8h7p\" (UID: \"b195d258-855f-47b7-a90f-9c1751cf0fe0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:22.181172 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178524 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-var-lib-openvswitch\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.181172 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178547 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-etc-sysctl-d\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.181172 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178570 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-sys\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.181172 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178596 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.181172 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178626 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-host-run-k8s-cni-cncf-io\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.181172 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178648 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/03659b3b-23b9-4ce3-9c69-d76c580fd1b7-serviceca\") pod \"node-ca-ldkxm\" (UID: \"03659b3b-23b9-4ce3-9c69-d76c580fd1b7\") " pod="openshift-image-registry/node-ca-ldkxm" Jul 06 19:48:22.181172 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178652 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbwjb\" (UniqueName: \"kubernetes.io/projected/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-kube-api-access-pbwjb\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.181172 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178700 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh4bc\" (UniqueName: \"kubernetes.io/projected/2df21624-1cb1-4437-99a7-b91b4784d32d-kube-api-access-wh4bc\") pod \"network-metrics-daemon-tgl28\" (UID: \"2df21624-1cb1-4437-99a7-b91b4784d32d\") " pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:22.181172 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178720 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-host\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.181658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178730 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-log-socket\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.181658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178773 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-multus-socket-dir-parent\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.181658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178798 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-log-socket\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.181658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178813 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03659b3b-23b9-4ce3-9c69-d76c580fd1b7-host\") pod \"node-ca-ldkxm\" (UID: \"03659b3b-23b9-4ce3-9c69-d76c580fd1b7\") " pod="openshift-image-registry/node-ca-ldkxm" Jul 06 19:48:22.181658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.178866 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ad86aa92-d126-4e69-a979-247c6deb986f-hosts-file\") pod \"node-resolver-gr86q\" (UID: \"ad86aa92-d126-4e69-a979-247c6deb986f\") " pod="openshift-dns/node-resolver-gr86q" Jul 06 19:48:22.181658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.179046 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-var-lib-openvswitch\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.181658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.179137 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-etc-sysctl-d\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.181658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.179187 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-sys\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.181658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.179285 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-cni-binary-copy\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.181658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.179404 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-etc-sysctl-conf\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.181658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.179478 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a59fb421-1d29-4686-8512-af1ee3473b27-konnectivity-ca\") pod \"konnectivity-agent-zwmlp\" (UID: \"a59fb421-1d29-4686-8512-af1ee3473b27\") " pod="kube-system/konnectivity-agent-zwmlp" Jul 06 19:48:22.181658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.179543 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0ffad6cc-ac00-4259-92dd-648660f40808-host-run-netns\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.181658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.179594 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-cnibin\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.181658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.179635 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-run\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.181658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.179685 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-multus-cni-dir\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.181658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.179740 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.181658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.179758 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-host-run-k8s-cni-cncf-io\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.181658 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.179803 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-host-var-lib-cni-bin\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.182199 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.180469 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-etc-tuned\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.182199 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.180520 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0ffad6cc-ac00-4259-92dd-648660f40808-ovn-node-metrics-cert\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.182199 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.180551 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-tmp\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.182199 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.180942 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a59fb421-1d29-4686-8512-af1ee3473b27-agent-certs\") pod \"konnectivity-agent-zwmlp\" (UID: \"a59fb421-1d29-4686-8512-af1ee3473b27\") " pod="kube-system/konnectivity-agent-zwmlp" Jul 06 19:48:22.186427 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.186408 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qwpw\" (UniqueName: \"kubernetes.io/projected/5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2-kube-api-access-5qwpw\") pod \"tuned-l58vr\" (UID: \"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2\") " pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.188263 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:22.188236 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:48:22.188388 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:22.188267 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:48:22.188388 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:22.188281 2581 projected.go:196] Error preparing data for projected volume kube-api-access-rjg8j for pod openshift-network-diagnostics/network-check-target-4wsff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:48:22.188388 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:22.188370 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j podName:7e42834f-b24c-4470-bf13-6351e2509148 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:22.688353071 +0000 UTC m=+2.135589783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rjg8j" (UniqueName: "kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j") pod "network-check-target-4wsff" (UID: "7e42834f-b24c-4470-bf13-6351e2509148") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:48:22.188998 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.188975 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh4bc\" (UniqueName: \"kubernetes.io/projected/2df21624-1cb1-4437-99a7-b91b4784d32d-kube-api-access-wh4bc\") pod \"network-metrics-daemon-tgl28\" (UID: \"2df21624-1cb1-4437-99a7-b91b4784d32d\") " pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:22.189649 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.189632 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbwjb\" (UniqueName: \"kubernetes.io/projected/ca05caa7-6d81-4a18-bc8e-d1070594cb2c-kube-api-access-pbwjb\") pod \"multus-cktbc\" (UID: \"ca05caa7-6d81-4a18-bc8e-d1070594cb2c\") " pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.189795 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.189770 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckvzv\" (UniqueName: \"kubernetes.io/projected/03659b3b-23b9-4ce3-9c69-d76c580fd1b7-kube-api-access-ckvzv\") pod \"node-ca-ldkxm\" (UID: \"03659b3b-23b9-4ce3-9c69-d76c580fd1b7\") " pod="openshift-image-registry/node-ca-ldkxm" Jul 06 19:48:22.190091 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.190045 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg49p\" (UniqueName: \"kubernetes.io/projected/88cb2c01-723f-4d6f-a264-8f6a110a1f6e-kube-api-access-rg49p\") pod \"multus-additional-cni-plugins-wlcd6\" (UID: \"88cb2c01-723f-4d6f-a264-8f6a110a1f6e\") " pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.190177 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.190103 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dgrz\" (UniqueName: \"kubernetes.io/projected/ad86aa92-d126-4e69-a979-247c6deb986f-kube-api-access-5dgrz\") pod \"node-resolver-gr86q\" (UID: \"ad86aa92-d126-4e69-a979-247c6deb986f\") " pod="openshift-dns/node-resolver-gr86q" Jul 06 19:48:22.190177 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.190126 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrf6q\" (UniqueName: \"kubernetes.io/projected/0ffad6cc-ac00-4259-92dd-648660f40808-kube-api-access-hrf6q\") pod \"ovnkube-node-sklff\" (UID: \"0ffad6cc-ac00-4259-92dd-648660f40808\") " pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.191155 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.191136 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt6bz\" (UniqueName: \"kubernetes.io/projected/d9cdca6a-9d67-4aae-980d-2d1800413281-kube-api-access-vt6bz\") pod \"iptables-alerter-cwnxx\" (UID: \"d9cdca6a-9d67-4aae-980d-2d1800413281\") " pod="openshift-network-operator/iptables-alerter-cwnxx" Jul 06 19:48:22.191384 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.191368 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc5sj\" (UniqueName: \"kubernetes.io/projected/b195d258-855f-47b7-a90f-9c1751cf0fe0-kube-api-access-xc5sj\") pod \"aws-ebs-csi-driver-node-w8h7p\" (UID: \"b195d258-855f-47b7-a90f-9c1751cf0fe0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:22.226734 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:22.226700 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8812863089e77e28db706ad567f2f4d4.slice/crio-d8144c926dff951ae873d6f11f720b3e94588c01c5176769d453caa2fb29a33f WatchSource:0}: Error finding container d8144c926dff951ae873d6f11f720b3e94588c01c5176769d453caa2fb29a33f: Status 404 returned error can't find the container with id d8144c926dff951ae873d6f11f720b3e94588c01c5176769d453caa2fb29a33f Jul 06 19:48:22.227173 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:22.227159 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8478f697752cbff42288357201271ac7.slice/crio-45aeceb98be7efd79eaf6cdcd77f2ba3628302cd66acd81280ba8eaa803cd4b4 WatchSource:0}: Error finding container 45aeceb98be7efd79eaf6cdcd77f2ba3628302cd66acd81280ba8eaa803cd4b4: Status 404 returned error can't find the container with id 45aeceb98be7efd79eaf6cdcd77f2ba3628302cd66acd81280ba8eaa803cd4b4 Jul 06 19:48:22.229010 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.228997 2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:48:22.231700 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.231687 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:48:22.303132 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.303104 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" Jul 06 19:48:22.310090 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.310069 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gr86q" Jul 06 19:48:22.313392 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:22.313357 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb195d258_855f_47b7_a90f_9c1751cf0fe0.slice/crio-b5c6a7ea4f6d996199ba96e3ec2c5e1f202fc34bf09bd9d2d2e558fe34a0ea48 WatchSource:0}: Error finding container b5c6a7ea4f6d996199ba96e3ec2c5e1f202fc34bf09bd9d2d2e558fe34a0ea48: Status 404 returned error can't find the container with id b5c6a7ea4f6d996199ba96e3ec2c5e1f202fc34bf09bd9d2d2e558fe34a0ea48 Jul 06 19:48:22.319166 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:22.319147 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad86aa92_d126_4e69_a979_247c6deb986f.slice/crio-ed35306b7da281b14345c12bf54c3c88da94b1bb558314acffffe6f03f91785f WatchSource:0}: Error finding container ed35306b7da281b14345c12bf54c3c88da94b1bb558314acffffe6f03f91785f: Status 404 returned error can't find the container with id ed35306b7da281b14345c12bf54c3c88da94b1bb558314acffffe6f03f91785f Jul 06 19:48:22.323625 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.323613 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ldkxm" Jul 06 19:48:22.331823 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:22.331805 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03659b3b_23b9_4ce3_9c69_d76c580fd1b7.slice/crio-04d0d2f2c99a331c25eaa65311c48ac16eb51f10a9dd32184066ed29e58ba4d3 WatchSource:0}: Error finding container 04d0d2f2c99a331c25eaa65311c48ac16eb51f10a9dd32184066ed29e58ba4d3: Status 404 returned error can't find the container with id 04d0d2f2c99a331c25eaa65311c48ac16eb51f10a9dd32184066ed29e58ba4d3 Jul 06 19:48:22.342432 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.342416 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:22.351191 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:22.351173 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ffad6cc_ac00_4259_92dd_648660f40808.slice/crio-4c28a6116857c9008a849e87e8ebd72e92d124d63aff970509dab8ac855a2ad9 WatchSource:0}: Error finding container 4c28a6116857c9008a849e87e8ebd72e92d124d63aff970509dab8ac855a2ad9: Status 404 returned error can't find the container with id 4c28a6116857c9008a849e87e8ebd72e92d124d63aff970509dab8ac855a2ad9 Jul 06 19:48:22.361378 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.361362 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zwmlp" Jul 06 19:48:22.369659 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:22.369641 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda59fb421_1d29_4686_8512_af1ee3473b27.slice/crio-10a8a4c9b5d1937e58c64be1374007c0cb2d602d8ee57c1762e5258d32110544 WatchSource:0}: Error finding container 10a8a4c9b5d1937e58c64be1374007c0cb2d602d8ee57c1762e5258d32110544: Status 404 returned error can't find the container with id 10a8a4c9b5d1937e58c64be1374007c0cb2d602d8ee57c1762e5258d32110544 Jul 06 19:48:22.380225 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.380204 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l58vr" Jul 06 19:48:22.387805 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:22.387786 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a0b1f1b_19ab_49d6_9c4e_38fa075bccb2.slice/crio-bb52b300f9eecb6ad49aa300268fe33b4ab67553732e2041a48ebee4746bd9e0 WatchSource:0}: Error finding container bb52b300f9eecb6ad49aa300268fe33b4ab67553732e2041a48ebee4746bd9e0: Status 404 returned error can't find the container with id bb52b300f9eecb6ad49aa300268fe33b4ab67553732e2041a48ebee4746bd9e0 Jul 06 19:48:22.399393 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.399372 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wlcd6" Jul 06 19:48:22.407267 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:22.407248 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88cb2c01_723f_4d6f_a264_8f6a110a1f6e.slice/crio-abacc5763eadf5124d65ed8775ced355d37d7f2c7fe8fd6e174bafcbd313d3a6 WatchSource:0}: Error finding container abacc5763eadf5124d65ed8775ced355d37d7f2c7fe8fd6e174bafcbd313d3a6: Status 404 returned error can't find the container with id abacc5763eadf5124d65ed8775ced355d37d7f2c7fe8fd6e174bafcbd313d3a6 Jul 06 19:48:22.413749 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.413733 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cktbc" Jul 06 19:48:22.419446 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.419429 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cwnxx" Jul 06 19:48:22.421192 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:22.421171 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca05caa7_6d81_4a18_bc8e_d1070594cb2c.slice/crio-cfefc13b8c45c50be81a2eea7ac8623e3e0d6f70fc545dca8234da5a28957fe9 WatchSource:0}: Error finding container cfefc13b8c45c50be81a2eea7ac8623e3e0d6f70fc545dca8234da5a28957fe9: Status 404 returned error can't find the container with id cfefc13b8c45c50be81a2eea7ac8623e3e0d6f70fc545dca8234da5a28957fe9 Jul 06 19:48:22.428251 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:22.428229 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9cdca6a_9d67_4aae_980d_2d1800413281.slice/crio-4ecc5456740e13522ffdc2469322c1557ac03700cfed926e268d36d8789711ed WatchSource:0}: Error finding container 4ecc5456740e13522ffdc2469322c1557ac03700cfed926e268d36d8789711ed: Status 404 returned error can't find the container with id 4ecc5456740e13522ffdc2469322c1557ac03700cfed926e268d36d8789711ed Jul 06 19:48:22.429906 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:22.429890 2581 kuberuntime_manager.go:1459] "Unhandled Error" err="container iptables-alerter start failed in pod iptables-alerter-cwnxx_openshift-network-operator(d9cdca6a-9d67-4aae-980d-2d1800413281): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 06 19:48:22.429954 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:22.429918 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"iptables-alerter\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-network-operator/iptables-alerter-cwnxx" podUID="d9cdca6a-9d67-4aae-980d-2d1800413281" Jul 06 19:48:22.682912 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.682823 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs\") pod \"network-metrics-daemon-tgl28\" (UID: \"2df21624-1cb1-4437-99a7-b91b4784d32d\") " pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:22.682912 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.682862 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret\") pod \"global-pull-secret-syncer-9qrkq\" (UID: \"baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6\") " pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:22.683116 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:22.682987 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:48:22.683116 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:22.683053 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs podName:2df21624-1cb1-4437-99a7-b91b4784d32d nodeName:}" failed. No retries permitted until 2026-07-06 19:48:23.683039119 +0000 UTC m=+3.130275815 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs") pod "network-metrics-daemon-tgl28" (UID: "2df21624-1cb1-4437-99a7-b91b4784d32d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:48:22.683116 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:22.682988 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:48:22.683116 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:22.683118 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret podName:baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:23.683106357 +0000 UTC m=+3.130343053 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret") pod "global-pull-secret-syncer-9qrkq" (UID: "baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:48:22.783612 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.783574 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjg8j\" (UniqueName: \"kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j\") pod \"network-check-target-4wsff\" (UID: \"7e42834f-b24c-4470-bf13-6351e2509148\") " pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:22.783790 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:22.783738 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:48:22.783790 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:22.783767 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:48:22.783790 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:22.783781 2581 projected.go:196] Error preparing data for projected volume kube-api-access-rjg8j for pod openshift-network-diagnostics/network-check-target-4wsff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:48:22.783919 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:22.783846 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j podName:7e42834f-b24c-4470-bf13-6351e2509148 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:23.783828436 +0000 UTC m=+3.231065134 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rjg8j" (UniqueName: "kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j") pod "network-check-target-4wsff" (UID: "7e42834f-b24c-4470-bf13-6351e2509148") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:48:22.939114 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:22.938843 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:48:23.019106 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:23.018996 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 19:43:22 +0000 UTC" deadline="2027-12-04 18:59:51.843546008 +0000 UTC" Jul 06 19:48:23.019106 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:23.019050 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12383h11m28.824500185s" Jul 06 19:48:23.091746 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:23.091107 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:23.091746 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:23.091223 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4wsff" podUID="7e42834f-b24c-4470-bf13-6351e2509148" Jul 06 19:48:23.101178 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:23.101111 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-172.ec2.internal" event={"ID":"8812863089e77e28db706ad567f2f4d4","Type":"ContainerStarted","Data":"d8144c926dff951ae873d6f11f720b3e94588c01c5176769d453caa2fb29a33f"} Jul 06 19:48:23.105601 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:23.105573 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cwnxx" event={"ID":"d9cdca6a-9d67-4aae-980d-2d1800413281","Type":"ContainerStarted","Data":"4ecc5456740e13522ffdc2469322c1557ac03700cfed926e268d36d8789711ed"} Jul 06 19:48:23.113307 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:23.112745 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"iptables-alerter\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:28389dd118629aa9d7c2525202f3ad6359afb98b6a3bf7672cb1ecfae2418e23\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-network-operator/iptables-alerter-cwnxx" podUID="d9cdca6a-9d67-4aae-980d-2d1800413281" Jul 06 19:48:23.115601 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:23.115530 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cktbc" event={"ID":"ca05caa7-6d81-4a18-bc8e-d1070594cb2c","Type":"ContainerStarted","Data":"cfefc13b8c45c50be81a2eea7ac8623e3e0d6f70fc545dca8234da5a28957fe9"} Jul 06 19:48:23.125331 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:23.125292 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l58vr" event={"ID":"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2","Type":"ContainerStarted","Data":"bb52b300f9eecb6ad49aa300268fe33b4ab67553732e2041a48ebee4746bd9e0"} Jul 06 19:48:23.127207 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:23.127183 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sklff" event={"ID":"0ffad6cc-ac00-4259-92dd-648660f40808","Type":"ContainerStarted","Data":"4c28a6116857c9008a849e87e8ebd72e92d124d63aff970509dab8ac855a2ad9"} Jul 06 19:48:23.133926 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:23.133901 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wlcd6" event={"ID":"88cb2c01-723f-4d6f-a264-8f6a110a1f6e","Type":"ContainerStarted","Data":"abacc5763eadf5124d65ed8775ced355d37d7f2c7fe8fd6e174bafcbd313d3a6"} Jul 06 19:48:23.137617 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:23.137592 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zwmlp" event={"ID":"a59fb421-1d29-4686-8512-af1ee3473b27","Type":"ContainerStarted","Data":"10a8a4c9b5d1937e58c64be1374007c0cb2d602d8ee57c1762e5258d32110544"} Jul 06 19:48:23.142428 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:23.142399 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ldkxm" event={"ID":"03659b3b-23b9-4ce3-9c69-d76c580fd1b7","Type":"ContainerStarted","Data":"04d0d2f2c99a331c25eaa65311c48ac16eb51f10a9dd32184066ed29e58ba4d3"} Jul 06 19:48:23.150621 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:23.150597 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gr86q" event={"ID":"ad86aa92-d126-4e69-a979-247c6deb986f","Type":"ContainerStarted","Data":"ed35306b7da281b14345c12bf54c3c88da94b1bb558314acffffe6f03f91785f"} Jul 06 19:48:23.157506 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:23.157480 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" event={"ID":"b195d258-855f-47b7-a90f-9c1751cf0fe0","Type":"ContainerStarted","Data":"b5c6a7ea4f6d996199ba96e3ec2c5e1f202fc34bf09bd9d2d2e558fe34a0ea48"} Jul 06 19:48:23.164271 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:23.164247 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-172.ec2.internal" event={"ID":"8478f697752cbff42288357201271ac7","Type":"ContainerStarted","Data":"45aeceb98be7efd79eaf6cdcd77f2ba3628302cd66acd81280ba8eaa803cd4b4"} Jul 06 19:48:23.690900 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:23.690866 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs\") pod \"network-metrics-daemon-tgl28\" (UID: \"2df21624-1cb1-4437-99a7-b91b4784d32d\") " pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:23.691096 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:23.690915 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret\") pod \"global-pull-secret-syncer-9qrkq\" (UID: \"baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6\") " pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:23.691096 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:23.691087 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:48:23.691216 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:23.691143 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret podName:baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:25.691125958 +0000 UTC m=+5.138362668 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret") pod "global-pull-secret-syncer-9qrkq" (UID: "baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:48:23.691960 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:23.691527 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:48:23.691960 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:23.691577 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs podName:2df21624-1cb1-4437-99a7-b91b4784d32d nodeName:}" failed. No retries permitted until 2026-07-06 19:48:25.69156175 +0000 UTC m=+5.138798449 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs") pod "network-metrics-daemon-tgl28" (UID: "2df21624-1cb1-4437-99a7-b91b4784d32d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:48:23.791803 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:23.791767 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjg8j\" (UniqueName: \"kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j\") pod \"network-check-target-4wsff\" (UID: \"7e42834f-b24c-4470-bf13-6351e2509148\") " pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:23.792028 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:23.791967 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:48:23.792028 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:23.791991 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:48:23.792028 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:23.792003 2581 projected.go:196] Error preparing data for projected volume kube-api-access-rjg8j for pod openshift-network-diagnostics/network-check-target-4wsff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:48:23.792194 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:23.792057 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j podName:7e42834f-b24c-4470-bf13-6351e2509148 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:25.792040665 +0000 UTC m=+5.239277384 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rjg8j" (UniqueName: "kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j") pod "network-check-target-4wsff" (UID: "7e42834f-b24c-4470-bf13-6351e2509148") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:48:24.020160 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:24.020069 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 19:43:22 +0000 UTC" deadline="2028-04-22 01:36:56.715404356 +0000 UTC" Jul 06 19:48:24.020160 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:24.020106 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15725h48m32.695302239s" Jul 06 19:48:24.089055 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:24.089019 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:24.089235 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:24.089161 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tgl28" podUID="2df21624-1cb1-4437-99a7-b91b4784d32d" Jul 06 19:48:24.089595 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:24.089574 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:24.089689 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:24.089674 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9qrkq" podUID="baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6" Jul 06 19:48:24.989943 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:24.989911 2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:48:25.092078 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:25.091464 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:25.092078 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:25.091589 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4wsff" podUID="7e42834f-b24c-4470-bf13-6351e2509148" Jul 06 19:48:25.710699 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:25.710547 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs\") pod \"network-metrics-daemon-tgl28\" (UID: \"2df21624-1cb1-4437-99a7-b91b4784d32d\") " pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:25.710699 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:25.710596 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret\") pod \"global-pull-secret-syncer-9qrkq\" (UID: \"baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6\") " pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:25.710939 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:25.710722 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:48:25.710939 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:25.710798 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs podName:2df21624-1cb1-4437-99a7-b91b4784d32d nodeName:}" failed. No retries permitted until 2026-07-06 19:48:29.710779546 +0000 UTC m=+9.158016255 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs") pod "network-metrics-daemon-tgl28" (UID: "2df21624-1cb1-4437-99a7-b91b4784d32d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:48:25.710939 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:25.710795 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:48:25.710939 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:25.710872 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret podName:baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:29.710849665 +0000 UTC m=+9.158086385 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret") pod "global-pull-secret-syncer-9qrkq" (UID: "baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:48:25.811061 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:25.811023 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjg8j\" (UniqueName: \"kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j\") pod \"network-check-target-4wsff\" (UID: \"7e42834f-b24c-4470-bf13-6351e2509148\") " pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:25.811255 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:25.811234 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:48:25.811356 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:25.811263 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:48:25.811356 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:25.811280 2581 projected.go:196] Error preparing data for projected volume kube-api-access-rjg8j for pod openshift-network-diagnostics/network-check-target-4wsff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:48:25.811467 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:25.811355 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j podName:7e42834f-b24c-4470-bf13-6351e2509148 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:29.811336161 +0000 UTC m=+9.258572867 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rjg8j" (UniqueName: "kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j") pod "network-check-target-4wsff" (UID: "7e42834f-b24c-4470-bf13-6351e2509148") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:48:26.090220 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:26.089635 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:26.090220 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:26.089772 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tgl28" podUID="2df21624-1cb1-4437-99a7-b91b4784d32d" Jul 06 19:48:26.090220 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:26.089635 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:26.090220 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:26.089873 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9qrkq" podUID="baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6" Jul 06 19:48:27.088711 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:27.088641 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:27.089157 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:27.088776 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4wsff" podUID="7e42834f-b24c-4470-bf13-6351e2509148" Jul 06 19:48:28.088865 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:28.088828 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:28.089347 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:28.088878 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:28.089347 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:28.088982 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9qrkq" podUID="baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6" Jul 06 19:48:28.089347 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:28.089121 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tgl28" podUID="2df21624-1cb1-4437-99a7-b91b4784d32d" Jul 06 19:48:29.089631 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:29.089602 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:29.090080 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:29.089726 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4wsff" podUID="7e42834f-b24c-4470-bf13-6351e2509148" Jul 06 19:48:29.744203 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:29.743644 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs\") pod \"network-metrics-daemon-tgl28\" (UID: \"2df21624-1cb1-4437-99a7-b91b4784d32d\") " pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:29.744203 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:29.743689 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret\") pod \"global-pull-secret-syncer-9qrkq\" (UID: \"baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6\") " pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:29.744203 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:29.743835 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:48:29.744203 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:29.743889 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret podName:baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:37.743872706 +0000 UTC m=+17.191109405 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret") pod "global-pull-secret-syncer-9qrkq" (UID: "baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:48:29.744203 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:29.743943 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:48:29.744203 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:29.743979 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs podName:2df21624-1cb1-4437-99a7-b91b4784d32d nodeName:}" failed. No retries permitted until 2026-07-06 19:48:37.743966809 +0000 UTC m=+17.191203521 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs") pod "network-metrics-daemon-tgl28" (UID: "2df21624-1cb1-4437-99a7-b91b4784d32d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:48:29.844929 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:29.844838 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjg8j\" (UniqueName: \"kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j\") pod \"network-check-target-4wsff\" (UID: \"7e42834f-b24c-4470-bf13-6351e2509148\") " pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:29.845101 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:29.844977 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:48:29.845101 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:29.845000 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:48:29.845101 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:29.845013 2581 projected.go:196] Error preparing data for projected volume kube-api-access-rjg8j for pod openshift-network-diagnostics/network-check-target-4wsff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:48:29.845101 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:29.845074 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j podName:7e42834f-b24c-4470-bf13-6351e2509148 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:37.845055071 +0000 UTC m=+17.292291769 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rjg8j" (UniqueName: "kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j") pod "network-check-target-4wsff" (UID: "7e42834f-b24c-4470-bf13-6351e2509148") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:48:30.089170 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:30.089127 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:30.089359 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:30.089171 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:30.089359 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:30.089275 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tgl28" podUID="2df21624-1cb1-4437-99a7-b91b4784d32d" Jul 06 19:48:30.089488 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:30.089411 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9qrkq" podUID="baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6" Jul 06 19:48:31.089161 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:31.089125 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:31.090965 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:31.090844 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4wsff" podUID="7e42834f-b24c-4470-bf13-6351e2509148" Jul 06 19:48:32.089455 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:32.089313 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:32.089919 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:32.089461 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9qrkq" podUID="baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6" Jul 06 19:48:32.089919 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:32.089498 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:32.089919 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:32.089597 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tgl28" podUID="2df21624-1cb1-4437-99a7-b91b4784d32d" Jul 06 19:48:33.089104 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:33.088851 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:33.089269 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:33.089235 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4wsff" podUID="7e42834f-b24c-4470-bf13-6351e2509148" Jul 06 19:48:34.088740 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:34.088707 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:34.089155 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:34.088710 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:34.089155 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:34.088834 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9qrkq" podUID="baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6" Jul 06 19:48:34.089155 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:34.088915 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tgl28" podUID="2df21624-1cb1-4437-99a7-b91b4784d32d" Jul 06 19:48:35.089228 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:35.089191 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:35.089708 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:35.089674 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4wsff" podUID="7e42834f-b24c-4470-bf13-6351e2509148" Jul 06 19:48:36.089070 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:36.089030 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:36.089260 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:36.089171 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9qrkq" podUID="baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6" Jul 06 19:48:36.089260 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:36.089225 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:36.089685 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:36.089357 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tgl28" podUID="2df21624-1cb1-4437-99a7-b91b4784d32d" Jul 06 19:48:37.089066 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:37.089027 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:37.089250 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:37.089157 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4wsff" podUID="7e42834f-b24c-4470-bf13-6351e2509148" Jul 06 19:48:37.801689 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:37.801653 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs\") pod \"network-metrics-daemon-tgl28\" (UID: \"2df21624-1cb1-4437-99a7-b91b4784d32d\") " pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:37.801689 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:37.801687 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret\") pod \"global-pull-secret-syncer-9qrkq\" (UID: \"baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6\") " pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:37.802093 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:37.801789 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:48:37.802093 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:37.801794 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:48:37.802093 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:37.801840 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret podName:baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:53.801823607 +0000 UTC m=+33.249060304 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret") pod "global-pull-secret-syncer-9qrkq" (UID: "baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:48:37.802093 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:37.801857 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs podName:2df21624-1cb1-4437-99a7-b91b4784d32d nodeName:}" failed. No retries permitted until 2026-07-06 19:48:53.801850113 +0000 UTC m=+33.249086809 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs") pod "network-metrics-daemon-tgl28" (UID: "2df21624-1cb1-4437-99a7-b91b4784d32d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:48:37.903016 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:37.902983 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjg8j\" (UniqueName: \"kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j\") pod \"network-check-target-4wsff\" (UID: \"7e42834f-b24c-4470-bf13-6351e2509148\") " pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:37.903159 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:37.903103 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:48:37.903159 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:37.903119 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:48:37.903159 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:37.903133 2581 projected.go:196] Error preparing data for projected volume kube-api-access-rjg8j for pod openshift-network-diagnostics/network-check-target-4wsff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:48:37.903254 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:37.903184 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j podName:7e42834f-b24c-4470-bf13-6351e2509148 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:53.903170977 +0000 UTC m=+33.350407672 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rjg8j" (UniqueName: "kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j") pod "network-check-target-4wsff" (UID: "7e42834f-b24c-4470-bf13-6351e2509148") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:48:38.088849 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:38.088775 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:38.088849 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:38.088814 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:38.089019 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:38.088922 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tgl28" podUID="2df21624-1cb1-4437-99a7-b91b4784d32d" Jul 06 19:48:38.089069 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:38.089047 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9qrkq" podUID="baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6" Jul 06 19:48:39.089641 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:39.089603 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:39.090096 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:39.089737 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4wsff" podUID="7e42834f-b24c-4470-bf13-6351e2509148" Jul 06 19:48:40.088806 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:40.088773 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:40.089001 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:40.088773 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:40.089001 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:40.088891 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tgl28" podUID="2df21624-1cb1-4437-99a7-b91b4784d32d" Jul 06 19:48:40.089001 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:40.088973 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9qrkq" podUID="baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6" Jul 06 19:48:41.090276 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:41.090241 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:41.090693 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:41.090369 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4wsff" podUID="7e42834f-b24c-4470-bf13-6351e2509148" Jul 06 19:48:42.089479 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:42.089449 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:42.089587 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:42.089568 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tgl28" podUID="2df21624-1cb1-4437-99a7-b91b4784d32d" Jul 06 19:48:42.089625 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:42.089601 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:42.089673 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:42.089660 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9qrkq" podUID="baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6" Jul 06 19:48:42.208769 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:42.208531 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sklff" event={"ID":"0ffad6cc-ac00-4259-92dd-648660f40808","Type":"ContainerStarted","Data":"7fb993ea690d186e1d7b4b4c6ebb47eea810f855ccde7c54eec8aaac42a14ed4"} Jul 06 19:48:42.209597 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:42.208820 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sklff" event={"ID":"0ffad6cc-ac00-4259-92dd-648660f40808","Type":"ContainerStarted","Data":"f19de7a152192d083f580f5bcfaa0780c365df22c73085c3be876e2b7ba0b446"} Jul 06 19:48:42.209912 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:42.209886 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-172.ec2.internal" event={"ID":"8812863089e77e28db706ad567f2f4d4","Type":"ContainerStarted","Data":"e7dbb85065f4bd1997c3a3304d496ec94b90ee093cc2a8a9371a26d299ea919e"} Jul 06 19:48:42.211369 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:42.211342 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cktbc" event={"ID":"ca05caa7-6d81-4a18-bc8e-d1070594cb2c","Type":"ContainerStarted","Data":"966039311e35f7fb0254658466fd5f99b70b09ed44a839c33ba54b62e97d457b"} Jul 06 19:48:42.212728 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:42.212708 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l58vr" event={"ID":"5a0b1f1b-19ab-49d6-9c4e-38fa075bccb2","Type":"ContainerStarted","Data":"fb54ed232da78c8fbd83a4cffed106a53fa66908f6a1fd2ebc6ce6525696860f"} Jul 06 19:48:42.224576 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:42.224534 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-172.ec2.internal" podStartSLOduration=21.224522236 podStartE2EDuration="21.224522236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:48:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:48:42.224179738 +0000 UTC m=+21.671416457" watchObservedRunningTime="2026-07-06 19:48:42.224522236 +0000 UTC m=+21.671758983" Jul 06 19:48:42.241851 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:42.241800 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-l58vr" podStartSLOduration=1.819151049 podStartE2EDuration="21.24178101s" totalImagesPullingTime="19.422629961s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:48:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:48:42.241629165 +0000 UTC m=+21.688865883" watchObservedRunningTime="2026-07-06 19:48:42.24178101 +0000 UTC m=+21.689017731" Jul 06 19:48:42.261020 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:42.260979 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cktbc" podStartSLOduration=1.813378101 podStartE2EDuration="21.260964459s" totalImagesPullingTime="19.447586358s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:48:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:48:42.260603762 +0000 UTC m=+21.707840683" watchObservedRunningTime="2026-07-06 19:48:42.260964459 +0000 UTC m=+21.708201177" Jul 06 19:48:43.089284 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:43.089210 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:43.089464 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:43.089314 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4wsff" podUID="7e42834f-b24c-4470-bf13-6351e2509148" Jul 06 19:48:43.217401 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:43.217184 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sklff" event={"ID":"0ffad6cc-ac00-4259-92dd-648660f40808","Type":"ContainerStarted","Data":"40dc16258944b11f5e76ca32f4ad5cc39354fb409b79668765832a440cdecc82"} Jul 06 19:48:43.217924 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:43.217454 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sklff" event={"ID":"0ffad6cc-ac00-4259-92dd-648660f40808","Type":"ContainerStarted","Data":"05be32ba43832767c8dbeac6bc4ad01460c629f722bced7a393718fe85fe90e8"} Jul 06 19:48:43.217924 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:43.217503 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sklff" event={"ID":"0ffad6cc-ac00-4259-92dd-648660f40808","Type":"ContainerStarted","Data":"a8b9f96243b7ad585431c9f932f3fcffa30500da923122987eb2846004dcb0c6"} Jul 06 19:48:43.217924 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:43.217549 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sklff" event={"ID":"0ffad6cc-ac00-4259-92dd-648660f40808","Type":"ContainerStarted","Data":"ef56c544bc802461ca5e9fca4bfdf74261e72536b348f56d0b793875d07f71b7"} Jul 06 19:48:43.218688 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:43.218660 2581 generic.go:356] "Generic (PLEG): container finished" podID="88cb2c01-723f-4d6f-a264-8f6a110a1f6e" containerID="11efb606533b055cae40523fdd32a2f39339d7b01ec412b23b05691825f8007c" exitCode=0 Jul 06 19:48:43.218806 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:43.218751 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wlcd6" event={"ID":"88cb2c01-723f-4d6f-a264-8f6a110a1f6e","Type":"ContainerDied","Data":"11efb606533b055cae40523fdd32a2f39339d7b01ec412b23b05691825f8007c"} Jul 06 19:48:43.220229 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:43.220203 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zwmlp" event={"ID":"a59fb421-1d29-4686-8512-af1ee3473b27","Type":"ContainerStarted","Data":"1e6135974018ef3c48a3e2a861411b1e7fd28c094ee2ba3ac8c18892b0cb55db"} Jul 06 19:48:43.221781 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:43.221758 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ldkxm" event={"ID":"03659b3b-23b9-4ce3-9c69-d76c580fd1b7","Type":"ContainerStarted","Data":"cf871410ccfcb3d2a6dbc11717a4292f7362045ffc45e4ce271c30490d7b18d7"} Jul 06 19:48:43.223117 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:43.223094 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gr86q" event={"ID":"ad86aa92-d126-4e69-a979-247c6deb986f","Type":"ContainerStarted","Data":"4e625a964a50e53bf5188dbe0c223cb408c3431f4e3aa00589defa551f956247"} Jul 06 19:48:43.224649 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:43.224620 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" event={"ID":"b195d258-855f-47b7-a90f-9c1751cf0fe0","Type":"ContainerStarted","Data":"7fdc2bc7fa781518db7b79e9c86d7e21edd517dabc1e60f0303a9ffb86b82572"} Jul 06 19:48:43.225965 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:43.225945 2581 generic.go:356] "Generic (PLEG): container finished" podID="8478f697752cbff42288357201271ac7" containerID="f41f054fa82b3d76804d5f9385bb128ca3c6c7197ee5578f637f216245a885c9" exitCode=0 Jul 06 19:48:43.226044 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:43.226022 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-172.ec2.internal" event={"ID":"8478f697752cbff42288357201271ac7","Type":"ContainerDied","Data":"f41f054fa82b3d76804d5f9385bb128ca3c6c7197ee5578f637f216245a885c9"} Jul 06 19:48:43.255204 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:43.255168 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zwmlp" podStartSLOduration=6.922286345 podStartE2EDuration="22.255154305s" totalImagesPullingTime="15.33286796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:48:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:48:43.254454466 +0000 UTC m=+22.701691181" watchObservedRunningTime="2026-07-06 19:48:43.255154305 +0000 UTC m=+22.702391023" Jul 06 19:48:43.284152 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:43.284102 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-gr86q" podStartSLOduration=2.794621471 podStartE2EDuration="22.284085818s" totalImagesPullingTime="19.489464347s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:48:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:48:43.283499173 +0000 UTC m=+22.730735892" watchObservedRunningTime="2026-07-06 19:48:43.284085818 +0000 UTC m=+22.731322534" Jul 06 19:48:43.298253 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:43.298215 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ldkxm" podStartSLOduration=2.8201160339999998 podStartE2EDuration="22.298201352s" totalImagesPullingTime="19.478085318s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:48:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:48:43.298027472 +0000 UTC m=+22.745264190" watchObservedRunningTime="2026-07-06 19:48:43.298201352 +0000 UTC m=+22.745438069" Jul 06 19:48:44.089137 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:44.089100 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:44.089137 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:44.089147 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:44.089451 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:44.089244 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tgl28" podUID="2df21624-1cb1-4437-99a7-b91b4784d32d" Jul 06 19:48:44.089451 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:44.089414 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9qrkq" podUID="baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6" Jul 06 19:48:44.230218 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:44.230186 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-172.ec2.internal" event={"ID":"8478f697752cbff42288357201271ac7","Type":"ContainerStarted","Data":"d63d5fa826c4c8913fdaa713eaeda215410b097965148a2a7bc58c3b0100a980"} Jul 06 19:48:44.231674 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:44.231655 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cwnxx" event={"ID":"d9cdca6a-9d67-4aae-980d-2d1800413281","Type":"ContainerStarted","Data":"6bca212f860a0d4b72fb3dfd9d1407a1a33d0148111f4b56b8bd5273096e02ca"} Jul 06 19:48:44.246891 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:44.246843 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-172.ec2.internal" podStartSLOduration=23.246828133 podStartE2EDuration="23.246828133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:48:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:48:44.246676489 +0000 UTC m=+23.693913211" watchObservedRunningTime="2026-07-06 19:48:44.246828133 +0000 UTC m=+23.694064850" Jul 06 19:48:44.262504 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:44.262461 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cwnxx" podStartSLOduration=23.262446409 podStartE2EDuration="23.262446409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:48:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-06 19:48:44.262168375 +0000 UTC m=+23.709405092" watchObservedRunningTime="2026-07-06 19:48:44.262446409 +0000 UTC m=+23.709683126" Jul 06 19:48:44.504235 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:44.504202 2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 19:48:44.592903 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:44.592871 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zwmlp" Jul 06 19:48:44.593669 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:44.593650 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zwmlp" Jul 06 19:48:45.054163 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:45.054021 2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T19:48:44.504228335Z","UUID":"92a0a5f2-9b79-4cdd-bba5-c3f0e946be6e","Handler":null,"Name":"","Endpoint":""} Jul 06 19:48:45.055552 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:45.055532 2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 19:48:45.055694 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:45.055572 2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 19:48:45.089051 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:45.089023 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:45.089216 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:45.089128 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4wsff" podUID="7e42834f-b24c-4470-bf13-6351e2509148" Jul 06 19:48:45.236478 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:45.236440 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" event={"ID":"b195d258-855f-47b7-a90f-9c1751cf0fe0","Type":"ContainerStarted","Data":"3de84a103908b4b86948b9141f8cfb0d030057c39de4d2f10997983b9ac26c23"} Jul 06 19:48:45.240278 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:45.240236 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sklff" event={"ID":"0ffad6cc-ac00-4259-92dd-648660f40808","Type":"ContainerStarted","Data":"e235e60b52f70d9b457a595c5ec595c6c9f494836e038484bbc0bc07bda84852"} Jul 06 19:48:46.089146 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:46.089058 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:46.089146 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:46.089096 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:46.089440 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:46.089177 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9qrkq" podUID="baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6" Jul 06 19:48:46.089440 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:46.089348 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tgl28" podUID="2df21624-1cb1-4437-99a7-b91b4784d32d" Jul 06 19:48:46.245394 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:46.245356 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" event={"ID":"b195d258-855f-47b7-a90f-9c1751cf0fe0","Type":"ContainerStarted","Data":"7a7de3c00222f66cfc7083502551560b5b9d055d15387089f0bd6b1e0e0c771d"} Jul 06 19:48:46.245394 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:46.245384 2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 19:48:46.263361 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:46.263291 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-w8h7p" podStartSLOduration=2.98317812 podStartE2EDuration="25.263272539s" totalImagesPullingTime="22.280094419s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:48:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:48:46.262684078 +0000 UTC m=+25.709920798" watchObservedRunningTime="2026-07-06 19:48:46.263272539 +0000 UTC m=+25.710509273" Jul 06 19:48:47.089732 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:47.089512 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:47.089885 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:47.089771 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4wsff" podUID="7e42834f-b24c-4470-bf13-6351e2509148" Jul 06 19:48:47.252091 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:47.252048 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-sklff" event={"ID":"0ffad6cc-ac00-4259-92dd-648660f40808","Type":"ContainerStarted","Data":"fa33028d6263d508391010e0582b56e4c1c9a1cd844c3df794564766a5a184c2"} Jul 06 19:48:47.282882 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:47.282828 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-sklff" podStartSLOduration=6.794351729 podStartE2EDuration="26.282811385s" totalImagesPullingTime="19.488459656s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:48:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:48:47.282794809 +0000 UTC m=+26.730031528" watchObservedRunningTime="2026-07-06 19:48:47.282811385 +0000 UTC m=+26.730048101" Jul 06 19:48:48.091146 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:48.090513 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:48.091146 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:48.090655 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tgl28" podUID="2df21624-1cb1-4437-99a7-b91b4784d32d" Jul 06 19:48:48.091146 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:48.091016 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:48.091146 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:48.091106 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9qrkq" podUID="baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6" Jul 06 19:48:48.255522 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:48.255490 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:48.255933 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:48.255582 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:48.255933 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:48.255633 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:48.273424 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:48.273363 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:48.273639 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:48.273619 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:48:48.773497 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:48.773465 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9qrkq"] Jul 06 19:48:48.773646 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:48.773574 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:48.773708 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:48.773676 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9qrkq" podUID="baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6" Jul 06 19:48:48.776700 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:48.776675 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tgl28"] Jul 06 19:48:48.776832 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:48.776773 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:48.776905 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:48.776880 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tgl28" podUID="2df21624-1cb1-4437-99a7-b91b4784d32d" Jul 06 19:48:48.780104 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:48.780081 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4wsff"] Jul 06 19:48:48.780203 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:48.780180 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:48.780300 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:48.780272 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4wsff" podUID="7e42834f-b24c-4470-bf13-6351e2509148" Jul 06 19:48:49.258902 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:49.258863 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wlcd6" event={"ID":"88cb2c01-723f-4d6f-a264-8f6a110a1f6e","Type":"ContainerStarted","Data":"232e0cbf451798e44d909c474ba3ad2582452b2c5446684d1880d9f0998b6132"} Jul 06 19:48:50.089121 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:50.088942 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:50.089284 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:50.089188 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4wsff" podUID="7e42834f-b24c-4470-bf13-6351e2509148" Jul 06 19:48:50.262922 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:50.262892 2581 generic.go:356] "Generic (PLEG): container finished" podID="88cb2c01-723f-4d6f-a264-8f6a110a1f6e" containerID="232e0cbf451798e44d909c474ba3ad2582452b2c5446684d1880d9f0998b6132" exitCode=0 Jul 06 19:48:50.263408 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:50.262964 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wlcd6" event={"ID":"88cb2c01-723f-4d6f-a264-8f6a110a1f6e","Type":"ContainerDied","Data":"232e0cbf451798e44d909c474ba3ad2582452b2c5446684d1880d9f0998b6132"} Jul 06 19:48:51.094751 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:51.094680 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:51.094888 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:51.094773 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9qrkq" podUID="baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6" Jul 06 19:48:51.094888 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:51.094823 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:51.094965 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:51.094918 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tgl28" podUID="2df21624-1cb1-4437-99a7-b91b4784d32d" Jul 06 19:48:51.751398 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:51.751370 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zwmlp" Jul 06 19:48:51.751721 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:51.751501 2581 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 19:48:51.752022 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:51.752000 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zwmlp" Jul 06 19:48:52.089539 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:52.089448 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:52.089675 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:52.089553 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4wsff" podUID="7e42834f-b24c-4470-bf13-6351e2509148" Jul 06 19:48:52.269799 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:52.269765 2581 generic.go:356] "Generic (PLEG): container finished" podID="88cb2c01-723f-4d6f-a264-8f6a110a1f6e" containerID="baf3c2588fe1e21123b4946e5fa9a695760ad6a8882455cd1021c07b353c367a" exitCode=0 Jul 06 19:48:52.269953 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:52.269833 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wlcd6" event={"ID":"88cb2c01-723f-4d6f-a264-8f6a110a1f6e","Type":"ContainerDied","Data":"baf3c2588fe1e21123b4946e5fa9a695760ad6a8882455cd1021c07b353c367a"} Jul 06 19:48:53.088951 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:53.088918 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:53.088951 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:53.088918 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:53.089376 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:53.089066 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tgl28" podUID="2df21624-1cb1-4437-99a7-b91b4784d32d" Jul 06 19:48:53.089376 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:53.089084 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9qrkq" podUID="baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6" Jul 06 19:48:53.821880 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:53.821636 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs\") pod \"network-metrics-daemon-tgl28\" (UID: \"2df21624-1cb1-4437-99a7-b91b4784d32d\") " pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:53.821880 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:53.821803 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:48:53.821880 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:53.821835 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret\") pod \"global-pull-secret-syncer-9qrkq\" (UID: \"baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6\") " pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:53.821880 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:53.821867 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs podName:2df21624-1cb1-4437-99a7-b91b4784d32d nodeName:}" failed. No retries permitted until 2026-07-06 19:49:25.821851279 +0000 UTC m=+65.269087975 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs") pod "network-metrics-daemon-tgl28" (UID: "2df21624-1cb1-4437-99a7-b91b4784d32d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:48:53.822138 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:53.821940 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:48:53.822138 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:53.821993 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret podName:baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:25.821978261 +0000 UTC m=+65.269214956 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret") pod "global-pull-secret-syncer-9qrkq" (UID: "baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:48:53.922716 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:53.922673 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjg8j\" (UniqueName: \"kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j\") pod \"network-check-target-4wsff\" (UID: \"7e42834f-b24c-4470-bf13-6351e2509148\") " pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:53.922889 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:53.922800 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:48:53.922889 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:53.922814 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:48:53.922889 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:53.922823 2581 projected.go:196] Error preparing data for projected volume kube-api-access-rjg8j for pod openshift-network-diagnostics/network-check-target-4wsff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:48:53.922889 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:53.922880 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j podName:7e42834f-b24c-4470-bf13-6351e2509148 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:25.922867339 +0000 UTC m=+65.370104036 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rjg8j" (UniqueName: "kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j") pod "network-check-target-4wsff" (UID: "7e42834f-b24c-4470-bf13-6351e2509148") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:48:54.089676 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.089583 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:54.090125 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:54.089708 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4wsff" podUID="7e42834f-b24c-4470-bf13-6351e2509148" Jul 06 19:48:54.277069 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.277035 2581 generic.go:356] "Generic (PLEG): container finished" podID="88cb2c01-723f-4d6f-a264-8f6a110a1f6e" containerID="db29658d9daf5e3220f764e3fe916a476de23c9a0b469ef8baf5668ebc38bcb0" exitCode=0 Jul 06 19:48:54.277215 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.277109 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wlcd6" event={"ID":"88cb2c01-723f-4d6f-a264-8f6a110a1f6e","Type":"ContainerDied","Data":"db29658d9daf5e3220f764e3fe916a476de23c9a0b469ef8baf5668ebc38bcb0"} Jul 06 19:48:54.862444 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.862358 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-172.ec2.internal" event="NodeReady" Jul 06 19:48:54.862587 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.862517 2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 19:48:54.899966 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.899935 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-65ddd84d6b-j244t"] Jul 06 19:48:54.919377 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.919345 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-r9czf"] Jul 06 19:48:54.919528 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.919442 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:54.922987 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.922964 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 19:48:54.923110 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.923040 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-g9x2b\"" Jul 06 19:48:54.923228 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.923211 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 19:48:54.923391 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.923371 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 19:48:54.929739 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.929713 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 19:48:54.933038 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.933018 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8jbg4"] Jul 06 19:48:54.933165 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.933149 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r9czf" Jul 06 19:48:54.935689 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.935527 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qb4h6\"" Jul 06 19:48:54.935689 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.935548 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 19:48:54.935689 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.935564 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 19:48:54.955185 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.955159 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65ddd84d6b-j244t"] Jul 06 19:48:54.955185 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.955194 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r9czf"] Jul 06 19:48:54.955379 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.955220 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8jbg4"] Jul 06 19:48:54.955379 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.955291 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8jbg4" Jul 06 19:48:54.957914 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.957879 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 19:48:54.957914 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.957907 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 19:48:54.958072 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.957940 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mdxh6\"" Jul 06 19:48:54.958072 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:54.958016 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 19:48:55.030892 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.030858 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19b3e6d4-9980-4130-bff7-3a9f41085938-tmp-dir\") pod \"dns-default-r9czf\" (UID: \"19b3e6d4-9980-4130-bff7-3a9f41085938\") " pod="openshift-dns/dns-default-r9czf" Jul 06 19:48:55.031042 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.030911 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/121e68c7-16d6-4c5a-a36b-9149e3561efc-image-registry-private-configuration\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.031042 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.030931 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8ba1d496-a1a9-4f60-a589-9ae3946d39b9-cert\") pod \"ingress-canary-8jbg4\" (UID: \"8ba1d496-a1a9-4f60-a589-9ae3946d39b9\") " pod="openshift-ingress-canary/ingress-canary-8jbg4" Jul 06 19:48:55.031042 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.031004 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/121e68c7-16d6-4c5a-a36b-9149e3561efc-trusted-ca\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.031042 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.031034 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-bound-sa-token\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.031174 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.031069 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/121e68c7-16d6-4c5a-a36b-9149e3561efc-installation-pull-secrets\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.031174 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.031104 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls\") pod \"dns-default-r9czf\" (UID: \"19b3e6d4-9980-4130-bff7-3a9f41085938\") " pod="openshift-dns/dns-default-r9czf" Jul 06 19:48:55.031174 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.031121 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.031174 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.031136 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-certificates\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.031174 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.031153 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjpjg\" (UniqueName: \"kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-kube-api-access-zjpjg\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.031313 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.031206 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/19b3e6d4-9980-4130-bff7-3a9f41085938-config-volume\") pod \"dns-default-r9czf\" (UID: \"19b3e6d4-9980-4130-bff7-3a9f41085938\") " pod="openshift-dns/dns-default-r9czf" Jul 06 19:48:55.031313 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.031232 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/121e68c7-16d6-4c5a-a36b-9149e3561efc-ca-trust-extracted\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.031313 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.031267 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwdg2\" (UniqueName: \"kubernetes.io/projected/19b3e6d4-9980-4130-bff7-3a9f41085938-kube-api-access-pwdg2\") pod \"dns-default-r9czf\" (UID: \"19b3e6d4-9980-4130-bff7-3a9f41085938\") " pod="openshift-dns/dns-default-r9czf" Jul 06 19:48:55.031313 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.031295 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4knfr\" (UniqueName: \"kubernetes.io/projected/8ba1d496-a1a9-4f60-a589-9ae3946d39b9-kube-api-access-4knfr\") pod \"ingress-canary-8jbg4\" (UID: \"8ba1d496-a1a9-4f60-a589-9ae3946d39b9\") " pod="openshift-ingress-canary/ingress-canary-8jbg4" Jul 06 19:48:55.089452 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.089417 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:48:55.089575 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.089465 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:48:55.092166 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.092142 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 19:48:55.092166 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.092152 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 19:48:55.092598 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.092177 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9xsp7\"" Jul 06 19:48:55.131948 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.131865 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls\") pod \"dns-default-r9czf\" (UID: \"19b3e6d4-9980-4130-bff7-3a9f41085938\") " pod="openshift-dns/dns-default-r9czf" Jul 06 19:48:55.131948 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.131909 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.131948 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.131940 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-certificates\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.132224 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.132019 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjpjg\" (UniqueName: \"kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-kube-api-access-zjpjg\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.132224 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.132056 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/19b3e6d4-9980-4130-bff7-3a9f41085938-config-volume\") pod \"dns-default-r9czf\" (UID: \"19b3e6d4-9980-4130-bff7-3a9f41085938\") " pod="openshift-dns/dns-default-r9czf" Jul 06 19:48:55.132224 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.132089 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/121e68c7-16d6-4c5a-a36b-9149e3561efc-ca-trust-extracted\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.132224 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:55.132101 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:48:55.132224 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.132115 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwdg2\" (UniqueName: \"kubernetes.io/projected/19b3e6d4-9980-4130-bff7-3a9f41085938-kube-api-access-pwdg2\") pod \"dns-default-r9czf\" (UID: \"19b3e6d4-9980-4130-bff7-3a9f41085938\") " pod="openshift-dns/dns-default-r9czf" Jul 06 19:48:55.132224 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.132145 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4knfr\" (UniqueName: \"kubernetes.io/projected/8ba1d496-a1a9-4f60-a589-9ae3946d39b9-kube-api-access-4knfr\") pod \"ingress-canary-8jbg4\" (UID: \"8ba1d496-a1a9-4f60-a589-9ae3946d39b9\") " pod="openshift-ingress-canary/ingress-canary-8jbg4" Jul 06 19:48:55.132224 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.132173 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19b3e6d4-9980-4130-bff7-3a9f41085938-tmp-dir\") pod \"dns-default-r9czf\" (UID: \"19b3e6d4-9980-4130-bff7-3a9f41085938\") " pod="openshift-dns/dns-default-r9czf" Jul 06 19:48:55.132224 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.132222 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/121e68c7-16d6-4c5a-a36b-9149e3561efc-image-registry-private-configuration\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.132625 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.132253 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8ba1d496-a1a9-4f60-a589-9ae3946d39b9-cert\") pod \"ingress-canary-8jbg4\" (UID: \"8ba1d496-a1a9-4f60-a589-9ae3946d39b9\") " pod="openshift-ingress-canary/ingress-canary-8jbg4" Jul 06 19:48:55.132625 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.132306 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/121e68c7-16d6-4c5a-a36b-9149e3561efc-trusted-ca\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.132625 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.132346 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-bound-sa-token\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.132625 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:55.132120 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65ddd84d6b-j244t: secret "image-registry-tls" not found Jul 06 19:48:55.132625 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.132614 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19b3e6d4-9980-4130-bff7-3a9f41085938-tmp-dir\") pod \"dns-default-r9czf\" (UID: \"19b3e6d4-9980-4130-bff7-3a9f41085938\") " pod="openshift-dns/dns-default-r9czf" Jul 06 19:48:55.132860 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:55.132664 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls podName:121e68c7-16d6-4c5a-a36b-9149e3561efc nodeName:}" failed. No retries permitted until 2026-07-06 19:48:55.632645372 +0000 UTC m=+35.079882087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls") pod "image-registry-65ddd84d6b-j244t" (UID: "121e68c7-16d6-4c5a-a36b-9149e3561efc") : secret "image-registry-tls" not found Jul 06 19:48:55.132860 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.132739 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/19b3e6d4-9980-4130-bff7-3a9f41085938-config-volume\") pod \"dns-default-r9czf\" (UID: \"19b3e6d4-9980-4130-bff7-3a9f41085938\") " pod="openshift-dns/dns-default-r9czf" Jul 06 19:48:55.132860 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.132790 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/121e68c7-16d6-4c5a-a36b-9149e3561efc-installation-pull-secrets\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.133014 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:55.132577 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:48:55.133014 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:55.132929 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls podName:19b3e6d4-9980-4130-bff7-3a9f41085938 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:55.632914186 +0000 UTC m=+35.080150892 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls") pod "dns-default-r9czf" (UID: "19b3e6d4-9980-4130-bff7-3a9f41085938") : secret "dns-default-metrics-tls" not found Jul 06 19:48:55.133179 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.133156 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/121e68c7-16d6-4c5a-a36b-9149e3561efc-ca-trust-extracted\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.133881 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.133849 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/121e68c7-16d6-4c5a-a36b-9149e3561efc-trusted-ca\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.136836 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.136814 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/121e68c7-16d6-4c5a-a36b-9149e3561efc-image-registry-private-configuration\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.136942 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.136848 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/121e68c7-16d6-4c5a-a36b-9149e3561efc-installation-pull-secrets\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.136942 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.136879 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8ba1d496-a1a9-4f60-a589-9ae3946d39b9-cert\") pod \"ingress-canary-8jbg4\" (UID: \"8ba1d496-a1a9-4f60-a589-9ae3946d39b9\") " pod="openshift-ingress-canary/ingress-canary-8jbg4" Jul 06 19:48:55.141512 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.141463 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-bound-sa-token\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.141512 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.141506 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwdg2\" (UniqueName: \"kubernetes.io/projected/19b3e6d4-9980-4130-bff7-3a9f41085938-kube-api-access-pwdg2\") pod \"dns-default-r9czf\" (UID: \"19b3e6d4-9980-4130-bff7-3a9f41085938\") " pod="openshift-dns/dns-default-r9czf" Jul 06 19:48:55.142053 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.142027 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjpjg\" (UniqueName: \"kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-kube-api-access-zjpjg\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.142269 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.142250 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4knfr\" (UniqueName: \"kubernetes.io/projected/8ba1d496-a1a9-4f60-a589-9ae3946d39b9-kube-api-access-4knfr\") pod \"ingress-canary-8jbg4\" (UID: \"8ba1d496-a1a9-4f60-a589-9ae3946d39b9\") " pod="openshift-ingress-canary/ingress-canary-8jbg4" Jul 06 19:48:55.144155 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.144137 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-certificates\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.265396 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.265363 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8jbg4" Jul 06 19:48:55.471170 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.471136 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8jbg4"] Jul 06 19:48:55.475669 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:48:55.475644 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ba1d496_a1a9_4f60_a589_9ae3946d39b9.slice/crio-d9a3631b3dbe95a0f52625b0bcdac1f1d6148946163620997ce1fb2d56ab7ab9 WatchSource:0}: Error finding container d9a3631b3dbe95a0f52625b0bcdac1f1d6148946163620997ce1fb2d56ab7ab9: Status 404 returned error can't find the container with id d9a3631b3dbe95a0f52625b0bcdac1f1d6148946163620997ce1fb2d56ab7ab9 Jul 06 19:48:55.636387 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.636281 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls\") pod \"dns-default-r9czf\" (UID: \"19b3e6d4-9980-4130-bff7-3a9f41085938\") " pod="openshift-dns/dns-default-r9czf" Jul 06 19:48:55.636387 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:55.636356 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:55.636613 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:55.636408 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:48:55.636613 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:55.636496 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls podName:19b3e6d4-9980-4130-bff7-3a9f41085938 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:56.636475278 +0000 UTC m=+36.083711989 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls") pod "dns-default-r9czf" (UID: "19b3e6d4-9980-4130-bff7-3a9f41085938") : secret "dns-default-metrics-tls" not found Jul 06 19:48:55.636613 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:55.636514 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:48:55.636613 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:55.636533 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65ddd84d6b-j244t: secret "image-registry-tls" not found Jul 06 19:48:55.636613 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:55.636579 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls podName:121e68c7-16d6-4c5a-a36b-9149e3561efc nodeName:}" failed. No retries permitted until 2026-07-06 19:48:56.636563683 +0000 UTC m=+36.083800385 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls") pod "image-registry-65ddd84d6b-j244t" (UID: "121e68c7-16d6-4c5a-a36b-9149e3561efc") : secret "image-registry-tls" not found Jul 06 19:48:56.089484 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:56.089448 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:48:56.092108 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:56.092082 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-t8fm7\"" Jul 06 19:48:56.092108 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:56.092080 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 19:48:56.093368 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:56.093349 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 19:48:56.282640 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:56.282605 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8jbg4" event={"ID":"8ba1d496-a1a9-4f60-a589-9ae3946d39b9","Type":"ContainerStarted","Data":"d9a3631b3dbe95a0f52625b0bcdac1f1d6148946163620997ce1fb2d56ab7ab9"} Jul 06 19:48:56.645455 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:56.645417 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls\") pod \"dns-default-r9czf\" (UID: \"19b3e6d4-9980-4130-bff7-3a9f41085938\") " pod="openshift-dns/dns-default-r9czf" Jul 06 19:48:56.645622 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:56.645469 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:56.645622 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:56.645576 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:48:56.645622 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:56.645606 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:48:56.645622 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:56.645622 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65ddd84d6b-j244t: secret "image-registry-tls" not found Jul 06 19:48:56.645838 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:56.645654 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls podName:19b3e6d4-9980-4130-bff7-3a9f41085938 nodeName:}" failed. No retries permitted until 2026-07-06 19:48:58.645634646 +0000 UTC m=+38.092871364 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls") pod "dns-default-r9czf" (UID: "19b3e6d4-9980-4130-bff7-3a9f41085938") : secret "dns-default-metrics-tls" not found Jul 06 19:48:56.645838 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:56.645675 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls podName:121e68c7-16d6-4c5a-a36b-9149e3561efc nodeName:}" failed. No retries permitted until 2026-07-06 19:48:58.645664768 +0000 UTC m=+38.092901481 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls") pod "image-registry-65ddd84d6b-j244t" (UID: "121e68c7-16d6-4c5a-a36b-9149e3561efc") : secret "image-registry-tls" not found Jul 06 19:48:58.663403 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:58.663111 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls\") pod \"dns-default-r9czf\" (UID: \"19b3e6d4-9980-4130-bff7-3a9f41085938\") " pod="openshift-dns/dns-default-r9czf" Jul 06 19:48:58.663403 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:58.663297 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:48:58.663403 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:48:58.663341 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:48:58.663403 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:58.663405 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls podName:19b3e6d4-9980-4130-bff7-3a9f41085938 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:02.663383195 +0000 UTC m=+42.110619893 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls") pod "dns-default-r9czf" (UID: "19b3e6d4-9980-4130-bff7-3a9f41085938") : secret "dns-default-metrics-tls" not found Jul 06 19:48:58.663908 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:58.663465 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:48:58.663908 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:58.663480 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65ddd84d6b-j244t: secret "image-registry-tls" not found Jul 06 19:48:58.663908 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:48:58.663549 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls podName:121e68c7-16d6-4c5a-a36b-9149e3561efc nodeName:}" failed. No retries permitted until 2026-07-06 19:49:02.663530858 +0000 UTC m=+42.110767558 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls") pod "image-registry-65ddd84d6b-j244t" (UID: "121e68c7-16d6-4c5a-a36b-9149e3561efc") : secret "image-registry-tls" not found Jul 06 19:49:01.298782 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:01.298751 2581 generic.go:356] "Generic (PLEG): container finished" podID="88cb2c01-723f-4d6f-a264-8f6a110a1f6e" containerID="4aa0ab0f1cbf4aa270c77f799e73f3524b232a766403e3b82fbcbb512df9c887" exitCode=0 Jul 06 19:49:01.299184 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:01.298837 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wlcd6" event={"ID":"88cb2c01-723f-4d6f-a264-8f6a110a1f6e","Type":"ContainerDied","Data":"4aa0ab0f1cbf4aa270c77f799e73f3524b232a766403e3b82fbcbb512df9c887"} Jul 06 19:49:01.300280 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:01.300261 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8jbg4" event={"ID":"8ba1d496-a1a9-4f60-a589-9ae3946d39b9","Type":"ContainerStarted","Data":"20aca993d5064ba6f7b9bda4ee9807c37c2e2732ed5218e8d5d56a8769766c4d"} Jul 06 19:49:01.338412 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:01.336415 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8jbg4" podStartSLOduration=2.574514633 podStartE2EDuration="7.336391647s" totalImagesPullingTime="4.761877014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:49:01.334153201 +0000 UTC m=+40.781389931" watchObservedRunningTime="2026-07-06 19:49:01.336391647 +0000 UTC m=+40.783628366" Jul 06 19:49:02.305207 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:02.305175 2581 generic.go:356] "Generic (PLEG): container finished" podID="88cb2c01-723f-4d6f-a264-8f6a110a1f6e" containerID="2b36f33807b6a9ca40ad963066e6ac6857dd12a4bf6176ed3a85aee6cd16b50f" exitCode=0 Jul 06 19:49:02.305601 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:02.305263 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wlcd6" event={"ID":"88cb2c01-723f-4d6f-a264-8f6a110a1f6e","Type":"ContainerDied","Data":"2b36f33807b6a9ca40ad963066e6ac6857dd12a4bf6176ed3a85aee6cd16b50f"} Jul 06 19:49:02.695073 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:02.694990 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls\") pod \"dns-default-r9czf\" (UID: \"19b3e6d4-9980-4130-bff7-3a9f41085938\") " pod="openshift-dns/dns-default-r9czf" Jul 06 19:49:02.695073 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:02.695041 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:49:02.695294 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:02.695099 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:49:02.695294 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:02.695137 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:49:02.695294 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:02.695149 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65ddd84d6b-j244t: secret "image-registry-tls" not found Jul 06 19:49:02.695294 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:02.695167 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls podName:19b3e6d4-9980-4130-bff7-3a9f41085938 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:10.695149845 +0000 UTC m=+50.142386552 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls") pod "dns-default-r9czf" (UID: "19b3e6d4-9980-4130-bff7-3a9f41085938") : secret "dns-default-metrics-tls" not found Jul 06 19:49:02.695294 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:02.695184 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls podName:121e68c7-16d6-4c5a-a36b-9149e3561efc nodeName:}" failed. No retries permitted until 2026-07-06 19:49:10.695174154 +0000 UTC m=+50.142410866 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls") pod "image-registry-65ddd84d6b-j244t" (UID: "121e68c7-16d6-4c5a-a36b-9149e3561efc") : secret "image-registry-tls" not found Jul 06 19:49:03.310925 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:03.310892 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wlcd6" event={"ID":"88cb2c01-723f-4d6f-a264-8f6a110a1f6e","Type":"ContainerStarted","Data":"0f43cab53c15faaa2908e748f7e4189ed99d2f2565bbc2f3ec85a07cfc2cb8ad"} Jul 06 19:49:03.336301 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:03.336246 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wlcd6" podStartSLOduration=8.720142601 podStartE2EDuration="42.336229572s" totalImagesPullingTime="33.616086971s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:48:21 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:49:03.334490881 +0000 UTC m=+42.781727613" watchObservedRunningTime="2026-07-06 19:49:03.336229572 +0000 UTC m=+42.783466289" Jul 06 19:49:10.757817 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:10.757780 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls\") pod \"dns-default-r9czf\" (UID: \"19b3e6d4-9980-4130-bff7-3a9f41085938\") " pod="openshift-dns/dns-default-r9czf" Jul 06 19:49:10.757817 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:10.757820 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:49:10.758231 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:10.757933 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:49:10.758231 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:10.757989 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:49:10.758231 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:10.758007 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65ddd84d6b-j244t: secret "image-registry-tls" not found Jul 06 19:49:10.758231 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:10.757998 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls podName:19b3e6d4-9980-4130-bff7-3a9f41085938 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:26.757981872 +0000 UTC m=+66.205218568 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls") pod "dns-default-r9czf" (UID: "19b3e6d4-9980-4130-bff7-3a9f41085938") : secret "dns-default-metrics-tls" not found Jul 06 19:49:10.758231 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:10.758057 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls podName:121e68c7-16d6-4c5a-a36b-9149e3561efc nodeName:}" failed. No retries permitted until 2026-07-06 19:49:26.758045456 +0000 UTC m=+66.205282155 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls") pod "image-registry-65ddd84d6b-j244t" (UID: "121e68c7-16d6-4c5a-a36b-9149e3561efc") : secret "image-registry-tls" not found Jul 06 19:49:20.273355 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:20.273315 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-sklff" Jul 06 19:49:24.828909 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.828876 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-2xj4r"] Jul 06 19:49:24.831359 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.831344 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" Jul 06 19:49:24.833956 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.833920 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:49:24.833956 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.833949 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-bdzmx\"" Jul 06 19:49:24.835049 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.835024 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 06 19:49:24.835161 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.835065 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 06 19:49:24.835161 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.835082 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 06 19:49:24.838936 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.838911 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 06 19:49:24.842915 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.842893 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-2xj4r"] Jul 06 19:49:24.862039 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.861998 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/25615b1c-46b5-484c-a198-88e9aac6fe21-trusted-ca\") pod \"console-operator-68b757fb67-2xj4r\" (UID: \"25615b1c-46b5-484c-a198-88e9aac6fe21\") " pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" Jul 06 19:49:24.862199 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.862053 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b892h\" (UniqueName: \"kubernetes.io/projected/25615b1c-46b5-484c-a198-88e9aac6fe21-kube-api-access-b892h\") pod \"console-operator-68b757fb67-2xj4r\" (UID: \"25615b1c-46b5-484c-a198-88e9aac6fe21\") " pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" Jul 06 19:49:24.862199 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.862134 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/25615b1c-46b5-484c-a198-88e9aac6fe21-config\") pod \"console-operator-68b757fb67-2xj4r\" (UID: \"25615b1c-46b5-484c-a198-88e9aac6fe21\") " pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" Jul 06 19:49:24.862199 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.862166 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/25615b1c-46b5-484c-a198-88e9aac6fe21-serving-cert\") pod \"console-operator-68b757fb67-2xj4r\" (UID: \"25615b1c-46b5-484c-a198-88e9aac6fe21\") " pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" Jul 06 19:49:24.924093 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.924061 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-79r5t"] Jul 06 19:49:24.925670 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.925654 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-79r5t" Jul 06 19:49:24.928167 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.928079 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-rs8d2\"" Jul 06 19:49:24.938689 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.938667 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-79r5t"] Jul 06 19:49:24.962814 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.962785 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/25615b1c-46b5-484c-a198-88e9aac6fe21-trusted-ca\") pod \"console-operator-68b757fb67-2xj4r\" (UID: \"25615b1c-46b5-484c-a198-88e9aac6fe21\") " pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" Jul 06 19:49:24.962956 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.962823 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b892h\" (UniqueName: \"kubernetes.io/projected/25615b1c-46b5-484c-a198-88e9aac6fe21-kube-api-access-b892h\") pod \"console-operator-68b757fb67-2xj4r\" (UID: \"25615b1c-46b5-484c-a198-88e9aac6fe21\") " pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" Jul 06 19:49:24.962956 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.962874 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/25615b1c-46b5-484c-a198-88e9aac6fe21-config\") pod \"console-operator-68b757fb67-2xj4r\" (UID: \"25615b1c-46b5-484c-a198-88e9aac6fe21\") " pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" Jul 06 19:49:24.962956 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.962894 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrlhv\" (UniqueName: \"kubernetes.io/projected/d87cec62-7a5c-4771-9a5e-79f7a2d8c5ac-kube-api-access-rrlhv\") pod \"network-check-source-989f8b8dc-79r5t\" (UID: \"d87cec62-7a5c-4771-9a5e-79f7a2d8c5ac\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-79r5t" Jul 06 19:49:24.962956 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.962914 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/25615b1c-46b5-484c-a198-88e9aac6fe21-serving-cert\") pod \"console-operator-68b757fb67-2xj4r\" (UID: \"25615b1c-46b5-484c-a198-88e9aac6fe21\") " pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" Jul 06 19:49:24.963531 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.963514 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/25615b1c-46b5-484c-a198-88e9aac6fe21-config\") pod \"console-operator-68b757fb67-2xj4r\" (UID: \"25615b1c-46b5-484c-a198-88e9aac6fe21\") " pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" Jul 06 19:49:24.963700 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.963682 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/25615b1c-46b5-484c-a198-88e9aac6fe21-trusted-ca\") pod \"console-operator-68b757fb67-2xj4r\" (UID: \"25615b1c-46b5-484c-a198-88e9aac6fe21\") " pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" Jul 06 19:49:24.966099 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.966078 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/25615b1c-46b5-484c-a198-88e9aac6fe21-serving-cert\") pod \"console-operator-68b757fb67-2xj4r\" (UID: \"25615b1c-46b5-484c-a198-88e9aac6fe21\") " pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" Jul 06 19:49:24.974168 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:24.974143 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b892h\" (UniqueName: \"kubernetes.io/projected/25615b1c-46b5-484c-a198-88e9aac6fe21-kube-api-access-b892h\") pod \"console-operator-68b757fb67-2xj4r\" (UID: \"25615b1c-46b5-484c-a198-88e9aac6fe21\") " pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" Jul 06 19:49:25.063511 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:25.063473 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrlhv\" (UniqueName: \"kubernetes.io/projected/d87cec62-7a5c-4771-9a5e-79f7a2d8c5ac-kube-api-access-rrlhv\") pod \"network-check-source-989f8b8dc-79r5t\" (UID: \"d87cec62-7a5c-4771-9a5e-79f7a2d8c5ac\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-79r5t" Jul 06 19:49:25.075392 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:25.075367 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 19:49:25.086037 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:25.085972 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 19:49:25.097297 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:25.097265 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrlhv\" (UniqueName: \"kubernetes.io/projected/d87cec62-7a5c-4771-9a5e-79f7a2d8c5ac-kube-api-access-rrlhv\") pod \"network-check-source-989f8b8dc-79r5t\" (UID: \"d87cec62-7a5c-4771-9a5e-79f7a2d8c5ac\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-79r5t" Jul 06 19:49:25.140367 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:25.140314 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" Jul 06 19:49:25.233875 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:25.233836 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-79r5t" Jul 06 19:49:25.296136 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:25.296108 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-2xj4r"] Jul 06 19:49:25.300937 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:49:25.300907 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25615b1c_46b5_484c_a198_88e9aac6fe21.slice/crio-aa03785a6f44f9072c235c3770fe27f632697e9659b2883ab0596ed3a3f98de2 WatchSource:0}: Error finding container aa03785a6f44f9072c235c3770fe27f632697e9659b2883ab0596ed3a3f98de2: Status 404 returned error can't find the container with id aa03785a6f44f9072c235c3770fe27f632697e9659b2883ab0596ed3a3f98de2 Jul 06 19:49:25.356015 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:25.355936 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-79r5t"] Jul 06 19:49:25.358758 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:49:25.358733 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd87cec62_7a5c_4771_9a5e_79f7a2d8c5ac.slice/crio-ff7dda7014f72569321c5e83693534b57549916a9b8abd775164a6493d1d9eca WatchSource:0}: Error finding container ff7dda7014f72569321c5e83693534b57549916a9b8abd775164a6493d1d9eca: Status 404 returned error can't find the container with id ff7dda7014f72569321c5e83693534b57549916a9b8abd775164a6493d1d9eca Jul 06 19:49:25.369064 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:25.369038 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" event={"ID":"25615b1c-46b5-484c-a198-88e9aac6fe21","Type":"ContainerStarted","Data":"aa03785a6f44f9072c235c3770fe27f632697e9659b2883ab0596ed3a3f98de2"} Jul 06 19:49:25.369934 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:25.369915 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-79r5t" event={"ID":"d87cec62-7a5c-4771-9a5e-79f7a2d8c5ac","Type":"ContainerStarted","Data":"ff7dda7014f72569321c5e83693534b57549916a9b8abd775164a6493d1d9eca"} Jul 06 19:49:25.869479 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:25.869438 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs\") pod \"network-metrics-daemon-tgl28\" (UID: \"2df21624-1cb1-4437-99a7-b91b4784d32d\") " pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:49:25.869479 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:25.869482 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret\") pod \"global-pull-secret-syncer-9qrkq\" (UID: \"baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6\") " pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:49:25.872443 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:25.872416 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 19:49:25.872443 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:25.872436 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 19:49:25.880642 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:25.880610 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 19:49:25.880793 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:25.880739 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs podName:2df21624-1cb1-4437-99a7-b91b4784d32d nodeName:}" failed. No retries permitted until 2026-07-06 19:50:29.880716186 +0000 UTC m=+129.327952883 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs") pod "network-metrics-daemon-tgl28" (UID: "2df21624-1cb1-4437-99a7-b91b4784d32d") : secret "metrics-daemon-secret" not found Jul 06 19:49:25.883480 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:25.883449 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6-original-pull-secret\") pod \"global-pull-secret-syncer-9qrkq\" (UID: \"baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6\") " pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:49:25.970802 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:25.970274 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjg8j\" (UniqueName: \"kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j\") pod \"network-check-target-4wsff\" (UID: \"7e42834f-b24c-4470-bf13-6351e2509148\") " pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:49:25.973538 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:25.973510 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjg8j\" (UniqueName: \"kubernetes.io/projected/7e42834f-b24c-4470-bf13-6351e2509148-kube-api-access-rjg8j\") pod \"network-check-target-4wsff\" (UID: \"7e42834f-b24c-4470-bf13-6351e2509148\") " pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:49:26.004280 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:26.004241 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9qrkq" Jul 06 19:49:26.102268 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:26.102111 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-t8fm7\"" Jul 06 19:49:26.110349 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:26.110274 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:49:26.157945 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:26.157904 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9qrkq"] Jul 06 19:49:26.162946 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:49:26.162910 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaaf2afb_f6b8_41e2_8ef5_a0e1e8ed03e6.slice/crio-54db410c1dc9bff4385baaca7036bdacf0673e3933d5618b388b3125e98b2845 WatchSource:0}: Error finding container 54db410c1dc9bff4385baaca7036bdacf0673e3933d5618b388b3125e98b2845: Status 404 returned error can't find the container with id 54db410c1dc9bff4385baaca7036bdacf0673e3933d5618b388b3125e98b2845 Jul 06 19:49:26.261293 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:26.261258 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4wsff"] Jul 06 19:49:26.264573 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:49:26.264542 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e42834f_b24c_4470_bf13_6351e2509148.slice/crio-e22e22a08b769c9a0cda35a26e6a6f3e4d82851b097a50db38c14fe78b3be29e WatchSource:0}: Error finding container e22e22a08b769c9a0cda35a26e6a6f3e4d82851b097a50db38c14fe78b3be29e: Status 404 returned error can't find the container with id e22e22a08b769c9a0cda35a26e6a6f3e4d82851b097a50db38c14fe78b3be29e Jul 06 19:49:26.373835 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:26.373793 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4wsff" event={"ID":"7e42834f-b24c-4470-bf13-6351e2509148","Type":"ContainerStarted","Data":"e22e22a08b769c9a0cda35a26e6a6f3e4d82851b097a50db38c14fe78b3be29e"} Jul 06 19:49:26.375011 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:26.374977 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9qrkq" event={"ID":"baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6","Type":"ContainerStarted","Data":"54db410c1dc9bff4385baaca7036bdacf0673e3933d5618b388b3125e98b2845"} Jul 06 19:49:26.775590 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:26.775548 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls\") pod \"dns-default-r9czf\" (UID: \"19b3e6d4-9980-4130-bff7-3a9f41085938\") " pod="openshift-dns/dns-default-r9czf" Jul 06 19:49:26.775766 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:26.775602 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:49:26.775766 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:26.775707 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:49:26.775766 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:26.775724 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65ddd84d6b-j244t: secret "image-registry-tls" not found Jul 06 19:49:26.775894 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:26.775777 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls podName:121e68c7-16d6-4c5a-a36b-9149e3561efc nodeName:}" failed. No retries permitted until 2026-07-06 19:49:58.775763316 +0000 UTC m=+98.223000025 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls") pod "image-registry-65ddd84d6b-j244t" (UID: "121e68c7-16d6-4c5a-a36b-9149e3561efc") : secret "image-registry-tls" not found Jul 06 19:49:26.775894 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:26.775706 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:49:26.775894 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:26.775856 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls podName:19b3e6d4-9980-4130-bff7-3a9f41085938 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:58.775838685 +0000 UTC m=+98.223075404 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls") pod "dns-default-r9czf" (UID: "19b3e6d4-9980-4130-bff7-3a9f41085938") : secret "dns-default-metrics-tls" not found Jul 06 19:49:28.385183 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:28.384175 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" event={"ID":"25615b1c-46b5-484c-a198-88e9aac6fe21","Type":"ContainerStarted","Data":"a3d7c246d4813e51113579c39b67cf19b845d1bda05464728dd1bee99d417aae"} Jul 06 19:49:28.385183 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:28.385092 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" Jul 06 19:49:28.386617 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:28.386312 2581 patch_prober.go:28] interesting pod/console-operator-68b757fb67-2xj4r container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.9:8443/readyz\": dial tcp 10.132.0.9:8443: connect: connection refused" start-of-body= Jul 06 19:49:28.386617 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:28.386387 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" podUID="25615b1c-46b5-484c-a198-88e9aac6fe21" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.9:8443/readyz\": dial tcp 10.132.0.9:8443: connect: connection refused" Jul 06 19:49:28.402039 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:28.401965 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" podStartSLOduration=1.771166065 podStartE2EDuration="4.401944402s" totalImagesPullingTime="2.630778337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:49:28.400773074 +0000 UTC m=+67.848009793" watchObservedRunningTime="2026-07-06 19:49:28.401944402 +0000 UTC m=+67.849181120" Jul 06 19:49:29.250597 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.250569 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lkt7q"] Jul 06 19:49:29.254775 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.254754 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:29.257514 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.257499 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 19:49:29.257596 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.257580 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 19:49:29.257711 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.257643 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 19:49:29.257804 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.257782 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 19:49:29.258730 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.258711 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2mgfw\"" Jul 06 19:49:29.271367 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.270491 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lkt7q"] Jul 06 19:49:29.297310 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.297278 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4ac0796b-7777-4b33-8a0e-a369697d8d88-crio-socket\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:29.297483 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.297316 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf4pg\" (UniqueName: \"kubernetes.io/projected/4ac0796b-7777-4b33-8a0e-a369697d8d88-kube-api-access-gf4pg\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:29.297483 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.297460 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ac0796b-7777-4b33-8a0e-a369697d8d88-data-volume\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:29.297631 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.297510 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4ac0796b-7777-4b33-8a0e-a369697d8d88-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:29.297631 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.297541 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4ac0796b-7777-4b33-8a0e-a369697d8d88-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:29.388661 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.388583 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/0.log" Jul 06 19:49:29.388661 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.388619 2581 generic.go:356] "Generic (PLEG): container finished" podID="25615b1c-46b5-484c-a198-88e9aac6fe21" containerID="a3d7c246d4813e51113579c39b67cf19b845d1bda05464728dd1bee99d417aae" exitCode=255 Jul 06 19:49:29.389121 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.388680 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" event={"ID":"25615b1c-46b5-484c-a198-88e9aac6fe21","Type":"ContainerDied","Data":"a3d7c246d4813e51113579c39b67cf19b845d1bda05464728dd1bee99d417aae"} Jul 06 19:49:29.389121 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.388949 2581 scope.go:117] "RemoveContainer" containerID="a3d7c246d4813e51113579c39b67cf19b845d1bda05464728dd1bee99d417aae" Jul 06 19:49:29.390285 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.390259 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4wsff" event={"ID":"7e42834f-b24c-4470-bf13-6351e2509148","Type":"ContainerStarted","Data":"a0ff8c07ed86720231e7b5b569b38c6c5aef4bfc6bec237db844b353bd643e21"} Jul 06 19:49:29.390420 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.390346 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:49:29.391813 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.391793 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-79r5t" event={"ID":"d87cec62-7a5c-4771-9a5e-79f7a2d8c5ac","Type":"ContainerStarted","Data":"7fae3799495f471fb74610d7a3e4eb9fe0cdaf520dce7b52d5319c2f0bdafef9"} Jul 06 19:49:29.398251 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.398233 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4ac0796b-7777-4b33-8a0e-a369697d8d88-crio-socket\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:29.398332 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.398261 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf4pg\" (UniqueName: \"kubernetes.io/projected/4ac0796b-7777-4b33-8a0e-a369697d8d88-kube-api-access-gf4pg\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:29.398332 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.398290 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ac0796b-7777-4b33-8a0e-a369697d8d88-data-volume\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:29.398332 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.398304 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4ac0796b-7777-4b33-8a0e-a369697d8d88-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:29.398508 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.398339 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4ac0796b-7777-4b33-8a0e-a369697d8d88-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:29.398508 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.398378 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4ac0796b-7777-4b33-8a0e-a369697d8d88-crio-socket\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:29.398508 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:29.398423 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:49:29.398508 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:29.398436 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:49:29.398508 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:29.398488 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4ac0796b-7777-4b33-8a0e-a369697d8d88-kube-rbac-proxy-cm podName:4ac0796b-7777-4b33-8a0e-a369697d8d88 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:29.898469581 +0000 UTC m=+69.345706302 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4ac0796b-7777-4b33-8a0e-a369697d8d88-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lkt7q" (UID: "4ac0796b-7777-4b33-8a0e-a369697d8d88") : configmap "kube-rbac-proxy" not found Jul 06 19:49:29.398677 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:29.398557 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ac0796b-7777-4b33-8a0e-a369697d8d88-insights-runtime-extractor-tls podName:4ac0796b-7777-4b33-8a0e-a369697d8d88 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:29.898538348 +0000 UTC m=+69.345775064 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4ac0796b-7777-4b33-8a0e-a369697d8d88-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lkt7q" (UID: "4ac0796b-7777-4b33-8a0e-a369697d8d88") : secret "insights-runtime-extractor-tls" not found Jul 06 19:49:29.398677 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.398620 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ac0796b-7777-4b33-8a0e-a369697d8d88-data-volume\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:29.416024 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.416002 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf4pg\" (UniqueName: \"kubernetes.io/projected/4ac0796b-7777-4b33-8a0e-a369697d8d88-kube-api-access-gf4pg\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:29.452533 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.452490 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-79r5t" podStartSLOduration=1.6158978739999998 podStartE2EDuration="5.45247392s" totalImagesPullingTime="3.836576046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:49:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:49:29.451713622 +0000 UTC m=+68.898950330" watchObservedRunningTime="2026-07-06 19:49:29.45247392 +0000 UTC m=+68.899710639" Jul 06 19:49:29.452668 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.452643 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4wsff" podStartSLOduration=65.50919032 podStartE2EDuration="1m8.452637671s" totalImagesPullingTime="2.943447351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:48:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:49:29.430496049 +0000 UTC m=+68.877732772" watchObservedRunningTime="2026-07-06 19:49:29.452637671 +0000 UTC m=+68.899874389" Jul 06 19:49:29.488265 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.488227 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-9jm7x"] Jul 06 19:49:29.491588 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.491566 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9jm7x" Jul 06 19:49:29.493958 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.493931 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 06 19:49:29.494087 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.493934 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 06 19:49:29.494149 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.493936 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-kxz9w\"" Jul 06 19:49:29.499072 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.499040 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-9jm7x"] Jul 06 19:49:29.600841 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.600810 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncqfg\" (UniqueName: \"kubernetes.io/projected/da05b35d-a058-47bc-bf23-e87c5fbbe9b9-kube-api-access-ncqfg\") pod \"migrator-d57d58cc9-9jm7x\" (UID: \"da05b35d-a058-47bc-bf23-e87c5fbbe9b9\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9jm7x" Jul 06 19:49:29.701769 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.701685 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncqfg\" (UniqueName: \"kubernetes.io/projected/da05b35d-a058-47bc-bf23-e87c5fbbe9b9-kube-api-access-ncqfg\") pod \"migrator-d57d58cc9-9jm7x\" (UID: \"da05b35d-a058-47bc-bf23-e87c5fbbe9b9\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9jm7x" Jul 06 19:49:29.711342 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.711298 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncqfg\" (UniqueName: \"kubernetes.io/projected/da05b35d-a058-47bc-bf23-e87c5fbbe9b9-kube-api-access-ncqfg\") pod \"migrator-d57d58cc9-9jm7x\" (UID: \"da05b35d-a058-47bc-bf23-e87c5fbbe9b9\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9jm7x" Jul 06 19:49:29.857669 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.857634 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9jm7x" Jul 06 19:49:29.903707 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.903077 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4ac0796b-7777-4b33-8a0e-a369697d8d88-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:29.903707 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.903129 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4ac0796b-7777-4b33-8a0e-a369697d8d88-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:29.903707 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:29.903286 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:49:29.903707 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:29.903369 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ac0796b-7777-4b33-8a0e-a369697d8d88-insights-runtime-extractor-tls podName:4ac0796b-7777-4b33-8a0e-a369697d8d88 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:30.903348123 +0000 UTC m=+70.350584821 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4ac0796b-7777-4b33-8a0e-a369697d8d88-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lkt7q" (UID: "4ac0796b-7777-4b33-8a0e-a369697d8d88") : secret "insights-runtime-extractor-tls" not found Jul 06 19:49:29.904133 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:29.903770 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:49:29.904133 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:29.903821 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4ac0796b-7777-4b33-8a0e-a369697d8d88-kube-rbac-proxy-cm podName:4ac0796b-7777-4b33-8a0e-a369697d8d88 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:30.903805121 +0000 UTC m=+70.351041817 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4ac0796b-7777-4b33-8a0e-a369697d8d88-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lkt7q" (UID: "4ac0796b-7777-4b33-8a0e-a369697d8d88") : configmap "kube-rbac-proxy" not found Jul 06 19:49:29.990022 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:29.989947 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-9jm7x"] Jul 06 19:49:29.992984 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:49:29.992954 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda05b35d_a058_47bc_bf23_e87c5fbbe9b9.slice/crio-e023102e790972a9098cbcf35f40ad3b605d23e24ea513239414a843986d1611 WatchSource:0}: Error finding container e023102e790972a9098cbcf35f40ad3b605d23e24ea513239414a843986d1611: Status 404 returned error can't find the container with id e023102e790972a9098cbcf35f40ad3b605d23e24ea513239414a843986d1611 Jul 06 19:49:30.396065 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:30.396032 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/1.log" Jul 06 19:49:30.396524 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:30.396503 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/0.log" Jul 06 19:49:30.396646 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:30.396542 2581 generic.go:356] "Generic (PLEG): container finished" podID="25615b1c-46b5-484c-a198-88e9aac6fe21" containerID="16418c58c23d86c6eab9ce3baab755a92025c48b4c8f063e2e38edf44eed82b7" exitCode=255 Jul 06 19:49:30.396646 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:30.396637 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" event={"ID":"25615b1c-46b5-484c-a198-88e9aac6fe21","Type":"ContainerDied","Data":"16418c58c23d86c6eab9ce3baab755a92025c48b4c8f063e2e38edf44eed82b7"} Jul 06 19:49:30.396754 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:30.396694 2581 scope.go:117] "RemoveContainer" containerID="a3d7c246d4813e51113579c39b67cf19b845d1bda05464728dd1bee99d417aae" Jul 06 19:49:30.397002 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:30.396978 2581 scope.go:117] "RemoveContainer" containerID="16418c58c23d86c6eab9ce3baab755a92025c48b4c8f063e2e38edf44eed82b7" Jul 06 19:49:30.397184 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:30.397153 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-2xj4r_openshift-console-operator(25615b1c-46b5-484c-a198-88e9aac6fe21)\"" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" podUID="25615b1c-46b5-484c-a198-88e9aac6fe21" Jul 06 19:49:30.398376 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:30.398353 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9jm7x" event={"ID":"da05b35d-a058-47bc-bf23-e87c5fbbe9b9","Type":"ContainerStarted","Data":"e023102e790972a9098cbcf35f40ad3b605d23e24ea513239414a843986d1611"} Jul 06 19:49:30.912884 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:30.912850 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4ac0796b-7777-4b33-8a0e-a369697d8d88-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:30.912884 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:30.912895 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4ac0796b-7777-4b33-8a0e-a369697d8d88-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:30.913165 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:30.912980 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:49:30.913165 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:30.913059 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4ac0796b-7777-4b33-8a0e-a369697d8d88-kube-rbac-proxy-cm podName:4ac0796b-7777-4b33-8a0e-a369697d8d88 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:32.913039638 +0000 UTC m=+72.360276359 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4ac0796b-7777-4b33-8a0e-a369697d8d88-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lkt7q" (UID: "4ac0796b-7777-4b33-8a0e-a369697d8d88") : configmap "kube-rbac-proxy" not found Jul 06 19:49:30.913165 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:30.913074 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:49:30.913165 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:30.913129 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ac0796b-7777-4b33-8a0e-a369697d8d88-insights-runtime-extractor-tls podName:4ac0796b-7777-4b33-8a0e-a369697d8d88 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:32.913113971 +0000 UTC m=+72.360350684 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4ac0796b-7777-4b33-8a0e-a369697d8d88-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lkt7q" (UID: "4ac0796b-7777-4b33-8a0e-a369697d8d88") : secret "insights-runtime-extractor-tls" not found Jul 06 19:49:31.236965 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:31.236886 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gr86q_ad86aa92-d126-4e69-a979-247c6deb986f/dns-node-resolver/0.log" Jul 06 19:49:31.401430 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:31.401400 2581 scope.go:117] "RemoveContainer" containerID="16418c58c23d86c6eab9ce3baab755a92025c48b4c8f063e2e38edf44eed82b7" Jul 06 19:49:31.401867 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:31.401578 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-2xj4r_openshift-console-operator(25615b1c-46b5-484c-a198-88e9aac6fe21)\"" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" podUID="25615b1c-46b5-484c-a198-88e9aac6fe21" Jul 06 19:49:32.232920 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:32.232897 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ldkxm_03659b3b-23b9-4ce3-9c69-d76c580fd1b7/node-ca/0.log" Jul 06 19:49:32.404856 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:32.404827 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/1.log" Jul 06 19:49:32.406574 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:32.406549 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9qrkq" event={"ID":"baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6","Type":"ContainerStarted","Data":"e5a1f188d206c766b999aba66bb996e3b07d78bdc27c0aa0cb752702d2607aa4"} Jul 06 19:49:32.424835 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:32.424789 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9qrkq" podStartSLOduration=65.982009157 podStartE2EDuration="1m11.424774579s" totalImagesPullingTime="5.442765422s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:48:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:49:32.424742863 +0000 UTC m=+71.871979594" watchObservedRunningTime="2026-07-06 19:49:32.424774579 +0000 UTC m=+71.872011298" Jul 06 19:49:32.832836 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:32.832767 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8jbg4_8ba1d496-a1a9-4f60-a589-9ae3946d39b9/serve-healthcheck-canary/0.log" Jul 06 19:49:32.930078 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:32.930053 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4ac0796b-7777-4b33-8a0e-a369697d8d88-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:32.930229 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:32.930088 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4ac0796b-7777-4b33-8a0e-a369697d8d88-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:32.930229 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:32.930174 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:49:32.930350 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:32.930234 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4ac0796b-7777-4b33-8a0e-a369697d8d88-kube-rbac-proxy-cm podName:4ac0796b-7777-4b33-8a0e-a369697d8d88 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:36.930221099 +0000 UTC m=+76.377457794 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4ac0796b-7777-4b33-8a0e-a369697d8d88-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lkt7q" (UID: "4ac0796b-7777-4b33-8a0e-a369697d8d88") : configmap "kube-rbac-proxy" not found Jul 06 19:49:32.930350 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:32.930265 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:49:32.930350 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:32.930311 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ac0796b-7777-4b33-8a0e-a369697d8d88-insights-runtime-extractor-tls podName:4ac0796b-7777-4b33-8a0e-a369697d8d88 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:36.930299302 +0000 UTC m=+76.377535999 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4ac0796b-7777-4b33-8a0e-a369697d8d88-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lkt7q" (UID: "4ac0796b-7777-4b33-8a0e-a369697d8d88") : secret "insights-runtime-extractor-tls" not found Jul 06 19:49:33.411585 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:33.411547 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9jm7x" event={"ID":"da05b35d-a058-47bc-bf23-e87c5fbbe9b9","Type":"ContainerStarted","Data":"18afe44e83fcf8835143115697b12fd84d307b4cdfa1afade4d9690ec100db01"} Jul 06 19:49:33.411932 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:33.411598 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9jm7x" event={"ID":"da05b35d-a058-47bc-bf23-e87c5fbbe9b9","Type":"ContainerStarted","Data":"7bf65377cf86857abbe71237f5edccad56b77e3cf335a0e81ec7fc2b36205afe"} Jul 06 19:49:33.431636 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:33.431587 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-9jm7x" podStartSLOduration=1.862187659 podStartE2EDuration="4.431571468s" totalImagesPullingTime="2.569383809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:49:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:49:33.430147242 +0000 UTC m=+72.877383960" watchObservedRunningTime="2026-07-06 19:49:33.431571468 +0000 UTC m=+72.878808186" Jul 06 19:49:35.141378 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:35.141347 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" Jul 06 19:49:35.141731 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:35.141654 2581 scope.go:117] "RemoveContainer" containerID="16418c58c23d86c6eab9ce3baab755a92025c48b4c8f063e2e38edf44eed82b7" Jul 06 19:49:35.141804 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:35.141786 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-2xj4r_openshift-console-operator(25615b1c-46b5-484c-a198-88e9aac6fe21)\"" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" podUID="25615b1c-46b5-484c-a198-88e9aac6fe21" Jul 06 19:49:36.958649 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:36.958601 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4ac0796b-7777-4b33-8a0e-a369697d8d88-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:36.958649 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:36.958646 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4ac0796b-7777-4b33-8a0e-a369697d8d88-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:36.959053 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:36.958731 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:49:36.959053 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:36.958798 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4ac0796b-7777-4b33-8a0e-a369697d8d88-kube-rbac-proxy-cm podName:4ac0796b-7777-4b33-8a0e-a369697d8d88 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:44.958780202 +0000 UTC m=+84.406016899 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4ac0796b-7777-4b33-8a0e-a369697d8d88-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lkt7q" (UID: "4ac0796b-7777-4b33-8a0e-a369697d8d88") : configmap "kube-rbac-proxy" not found Jul 06 19:49:36.959053 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:36.958801 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:49:36.959053 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:36.958841 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ac0796b-7777-4b33-8a0e-a369697d8d88-insights-runtime-extractor-tls podName:4ac0796b-7777-4b33-8a0e-a369697d8d88 nodeName:}" failed. No retries permitted until 2026-07-06 19:49:44.958830232 +0000 UTC m=+84.406066928 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4ac0796b-7777-4b33-8a0e-a369697d8d88-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lkt7q" (UID: "4ac0796b-7777-4b33-8a0e-a369697d8d88") : secret "insights-runtime-extractor-tls" not found Jul 06 19:49:38.385469 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:38.385433 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" Jul 06 19:49:38.385918 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:38.385784 2581 scope.go:117] "RemoveContainer" containerID="16418c58c23d86c6eab9ce3baab755a92025c48b4c8f063e2e38edf44eed82b7" Jul 06 19:49:38.385957 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:38.385916 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-2xj4r_openshift-console-operator(25615b1c-46b5-484c-a198-88e9aac6fe21)\"" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" podUID="25615b1c-46b5-484c-a198-88e9aac6fe21" Jul 06 19:49:45.017205 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:45.017033 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4ac0796b-7777-4b33-8a0e-a369697d8d88-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:45.017205 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:45.017108 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4ac0796b-7777-4b33-8a0e-a369697d8d88-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:45.017662 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:45.017271 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:49:45.017662 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:45.017369 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ac0796b-7777-4b33-8a0e-a369697d8d88-insights-runtime-extractor-tls podName:4ac0796b-7777-4b33-8a0e-a369697d8d88 nodeName:}" failed. No retries permitted until 2026-07-06 19:50:01.017348417 +0000 UTC m=+100.464585114 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4ac0796b-7777-4b33-8a0e-a369697d8d88-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lkt7q" (UID: "4ac0796b-7777-4b33-8a0e-a369697d8d88") : secret "insights-runtime-extractor-tls" not found Jul 06 19:49:45.017743 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:45.017682 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4ac0796b-7777-4b33-8a0e-a369697d8d88-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:49:54.089253 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:54.089220 2581 scope.go:117] "RemoveContainer" containerID="16418c58c23d86c6eab9ce3baab755a92025c48b4c8f063e2e38edf44eed82b7" Jul 06 19:49:54.475228 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:54.475155 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/2.log" Jul 06 19:49:54.475537 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:54.475522 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/1.log" Jul 06 19:49:54.475592 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:54.475553 2581 generic.go:356] "Generic (PLEG): container finished" podID="25615b1c-46b5-484c-a198-88e9aac6fe21" containerID="94d4e26e892f13ae8baaaaaa24925cddea651a7130f6f44e736e04045a676987" exitCode=255 Jul 06 19:49:54.475592 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:54.475582 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" event={"ID":"25615b1c-46b5-484c-a198-88e9aac6fe21","Type":"ContainerDied","Data":"94d4e26e892f13ae8baaaaaa24925cddea651a7130f6f44e736e04045a676987"} Jul 06 19:49:54.475682 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:54.475623 2581 scope.go:117] "RemoveContainer" containerID="16418c58c23d86c6eab9ce3baab755a92025c48b4c8f063e2e38edf44eed82b7" Jul 06 19:49:54.475931 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:54.475908 2581 scope.go:117] "RemoveContainer" containerID="94d4e26e892f13ae8baaaaaa24925cddea651a7130f6f44e736e04045a676987" Jul 06 19:49:54.476099 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:54.476081 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-2xj4r_openshift-console-operator(25615b1c-46b5-484c-a198-88e9aac6fe21)\"" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" podUID="25615b1c-46b5-484c-a198-88e9aac6fe21" Jul 06 19:49:55.141298 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:55.141261 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" Jul 06 19:49:55.480281 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:55.480200 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/2.log" Jul 06 19:49:55.480567 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:55.480552 2581 scope.go:117] "RemoveContainer" containerID="94d4e26e892f13ae8baaaaaa24925cddea651a7130f6f44e736e04045a676987" Jul 06 19:49:55.480725 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:55.480707 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-2xj4r_openshift-console-operator(25615b1c-46b5-484c-a198-88e9aac6fe21)\"" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" podUID="25615b1c-46b5-484c-a198-88e9aac6fe21" Jul 06 19:49:57.108527 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:57.108496 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b564cbdf-6t5b9"] Jul 06 19:49:57.112100 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:57.112081 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b564cbdf-6t5b9" Jul 06 19:49:57.118312 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:57.118288 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 06 19:49:57.118418 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:57.118286 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 19:49:57.118476 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:57.118441 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 19:49:57.118546 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:57.118528 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 19:49:57.129895 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:57.129870 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b564cbdf-6t5b9"] Jul 06 19:49:57.205306 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:57.205272 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/979fcd4b-bbbd-4ce5-b5fa-e6f6da95578d-klusterlet-config\") pod \"klusterlet-addon-workmgr-78b564cbdf-6t5b9\" (UID: \"979fcd4b-bbbd-4ce5-b5fa-e6f6da95578d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b564cbdf-6t5b9" Jul 06 19:49:57.205484 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:57.205438 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/979fcd4b-bbbd-4ce5-b5fa-e6f6da95578d-tmp\") pod \"klusterlet-addon-workmgr-78b564cbdf-6t5b9\" (UID: \"979fcd4b-bbbd-4ce5-b5fa-e6f6da95578d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b564cbdf-6t5b9" Jul 06 19:49:57.205484 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:57.205474 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vjhr\" (UniqueName: \"kubernetes.io/projected/979fcd4b-bbbd-4ce5-b5fa-e6f6da95578d-kube-api-access-6vjhr\") pod \"klusterlet-addon-workmgr-78b564cbdf-6t5b9\" (UID: \"979fcd4b-bbbd-4ce5-b5fa-e6f6da95578d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b564cbdf-6t5b9" Jul 06 19:49:57.306803 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:57.306770 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vjhr\" (UniqueName: \"kubernetes.io/projected/979fcd4b-bbbd-4ce5-b5fa-e6f6da95578d-kube-api-access-6vjhr\") pod \"klusterlet-addon-workmgr-78b564cbdf-6t5b9\" (UID: \"979fcd4b-bbbd-4ce5-b5fa-e6f6da95578d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b564cbdf-6t5b9" Jul 06 19:49:57.306989 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:57.306811 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/979fcd4b-bbbd-4ce5-b5fa-e6f6da95578d-klusterlet-config\") pod \"klusterlet-addon-workmgr-78b564cbdf-6t5b9\" (UID: \"979fcd4b-bbbd-4ce5-b5fa-e6f6da95578d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b564cbdf-6t5b9" Jul 06 19:49:57.307055 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:57.307013 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/979fcd4b-bbbd-4ce5-b5fa-e6f6da95578d-tmp\") pod \"klusterlet-addon-workmgr-78b564cbdf-6t5b9\" (UID: \"979fcd4b-bbbd-4ce5-b5fa-e6f6da95578d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b564cbdf-6t5b9" Jul 06 19:49:57.307381 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:57.307364 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/979fcd4b-bbbd-4ce5-b5fa-e6f6da95578d-tmp\") pod \"klusterlet-addon-workmgr-78b564cbdf-6t5b9\" (UID: \"979fcd4b-bbbd-4ce5-b5fa-e6f6da95578d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b564cbdf-6t5b9" Jul 06 19:49:57.309119 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:57.309104 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/979fcd4b-bbbd-4ce5-b5fa-e6f6da95578d-klusterlet-config\") pod \"klusterlet-addon-workmgr-78b564cbdf-6t5b9\" (UID: \"979fcd4b-bbbd-4ce5-b5fa-e6f6da95578d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b564cbdf-6t5b9" Jul 06 19:49:57.317727 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:57.317702 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vjhr\" (UniqueName: \"kubernetes.io/projected/979fcd4b-bbbd-4ce5-b5fa-e6f6da95578d-kube-api-access-6vjhr\") pod \"klusterlet-addon-workmgr-78b564cbdf-6t5b9\" (UID: \"979fcd4b-bbbd-4ce5-b5fa-e6f6da95578d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b564cbdf-6t5b9" Jul 06 19:49:57.421458 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:57.421377 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b564cbdf-6t5b9" Jul 06 19:49:57.547563 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:57.547525 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b564cbdf-6t5b9"] Jul 06 19:49:57.550828 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:49:57.550803 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod979fcd4b_bbbd_4ce5_b5fa_e6f6da95578d.slice/crio-ff1ca48b97fb8adb2d79a6d431a8c01dd570d626ebf423b83b64ea1ddbb2d44b WatchSource:0}: Error finding container ff1ca48b97fb8adb2d79a6d431a8c01dd570d626ebf423b83b64ea1ddbb2d44b: Status 404 returned error can't find the container with id ff1ca48b97fb8adb2d79a6d431a8c01dd570d626ebf423b83b64ea1ddbb2d44b Jul 06 19:49:58.386524 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:58.385829 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" Jul 06 19:49:58.386524 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:58.386241 2581 scope.go:117] "RemoveContainer" containerID="94d4e26e892f13ae8baaaaaa24925cddea651a7130f6f44e736e04045a676987" Jul 06 19:49:58.386524 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:49:58.386459 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-2xj4r_openshift-console-operator(25615b1c-46b5-484c-a198-88e9aac6fe21)\"" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" podUID="25615b1c-46b5-484c-a198-88e9aac6fe21" Jul 06 19:49:58.494806 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:58.494733 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b564cbdf-6t5b9" event={"ID":"979fcd4b-bbbd-4ce5-b5fa-e6f6da95578d","Type":"ContainerStarted","Data":"ff1ca48b97fb8adb2d79a6d431a8c01dd570d626ebf423b83b64ea1ddbb2d44b"} Jul 06 19:49:58.818142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:58.818104 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:49:58.818341 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:58.818156 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls\") pod \"dns-default-r9czf\" (UID: \"19b3e6d4-9980-4130-bff7-3a9f41085938\") " pod="openshift-dns/dns-default-r9czf" Jul 06 19:49:58.821384 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:58.821360 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls\") pod \"image-registry-65ddd84d6b-j244t\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:49:58.822251 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:58.822228 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/19b3e6d4-9980-4130-bff7-3a9f41085938-metrics-tls\") pod \"dns-default-r9czf\" (UID: \"19b3e6d4-9980-4130-bff7-3a9f41085938\") " pod="openshift-dns/dns-default-r9czf" Jul 06 19:49:58.833273 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:58.833253 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-g9x2b\"" Jul 06 19:49:58.841252 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:58.841217 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:49:58.845971 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:58.845944 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qb4h6\"" Jul 06 19:49:58.854988 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:58.854784 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r9czf" Jul 06 19:49:59.011449 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:59.011416 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65ddd84d6b-j244t"] Jul 06 19:49:59.017103 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:49:59.017059 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod121e68c7_16d6_4c5a_a36b_9149e3561efc.slice/crio-27a08bd3a28a93e9d07ae1f25fc3e7222cd76ea52eb5765e22cda8c1013afd07 WatchSource:0}: Error finding container 27a08bd3a28a93e9d07ae1f25fc3e7222cd76ea52eb5765e22cda8c1013afd07: Status 404 returned error can't find the container with id 27a08bd3a28a93e9d07ae1f25fc3e7222cd76ea52eb5765e22cda8c1013afd07 Jul 06 19:49:59.035863 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:59.035835 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r9czf"] Jul 06 19:49:59.040741 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:49:59.040700 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19b3e6d4_9980_4130_bff7_3a9f41085938.slice/crio-28111b7393b6e4afbe41d6800f42b748ddc4c7055f0b58f8bd41717a9fe6bfb8 WatchSource:0}: Error finding container 28111b7393b6e4afbe41d6800f42b748ddc4c7055f0b58f8bd41717a9fe6bfb8: Status 404 returned error can't find the container with id 28111b7393b6e4afbe41d6800f42b748ddc4c7055f0b58f8bd41717a9fe6bfb8 Jul 06 19:49:59.500683 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:59.500553 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r9czf" event={"ID":"19b3e6d4-9980-4130-bff7-3a9f41085938","Type":"ContainerStarted","Data":"28111b7393b6e4afbe41d6800f42b748ddc4c7055f0b58f8bd41717a9fe6bfb8"} Jul 06 19:49:59.502581 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:59.502535 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" event={"ID":"121e68c7-16d6-4c5a-a36b-9149e3561efc","Type":"ContainerStarted","Data":"d2048ff618a49c90da922ef8988e952136af12fc0dec07aedfb21c66cbe2eca4"} Jul 06 19:49:59.502777 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:59.502585 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" event={"ID":"121e68c7-16d6-4c5a-a36b-9149e3561efc","Type":"ContainerStarted","Data":"27a08bd3a28a93e9d07ae1f25fc3e7222cd76ea52eb5765e22cda8c1013afd07"} Jul 06 19:49:59.502777 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:59.502689 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:49:59.553400 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:59.553339 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" podStartSLOduration=98.553301841 podStartE2EDuration="1m38.553301841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:48:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:49:59.553151755 +0000 UTC m=+99.000388473" watchObservedRunningTime="2026-07-06 19:49:59.553301841 +0000 UTC m=+99.000538596" Jul 06 19:49:59.598248 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:59.598215 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-794xr"] Jul 06 19:49:59.608475 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:59.608439 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-794xr" Jul 06 19:49:59.611249 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:59.611222 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 06 19:49:59.611249 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:59.611240 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-cjjch\"" Jul 06 19:49:59.617431 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:59.617207 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-794xr"] Jul 06 19:49:59.725358 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:59.725308 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ab738938-c8aa-4c7d-b230-6e0b385079d6-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-794xr\" (UID: \"ab738938-c8aa-4c7d-b230-6e0b385079d6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-794xr" Jul 06 19:49:59.826667 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:59.826635 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ab738938-c8aa-4c7d-b230-6e0b385079d6-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-794xr\" (UID: \"ab738938-c8aa-4c7d-b230-6e0b385079d6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-794xr" Jul 06 19:49:59.829749 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:59.829695 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ab738938-c8aa-4c7d-b230-6e0b385079d6-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-794xr\" (UID: \"ab738938-c8aa-4c7d-b230-6e0b385079d6\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-794xr" Jul 06 19:49:59.920959 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:49:59.920927 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-794xr" Jul 06 19:50:00.181495 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:00.181467 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-794xr"] Jul 06 19:50:00.184661 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:50:00.184618 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab738938_c8aa_4c7d_b230_6e0b385079d6.slice/crio-c8ed748154ac4e6753aba44b789e0114fa988727f24e1e672fbccb84c576f845 WatchSource:0}: Error finding container c8ed748154ac4e6753aba44b789e0114fa988727f24e1e672fbccb84c576f845: Status 404 returned error can't find the container with id c8ed748154ac4e6753aba44b789e0114fa988727f24e1e672fbccb84c576f845 Jul 06 19:50:00.401167 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:00.401091 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4wsff" Jul 06 19:50:00.507331 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:00.507279 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-794xr" event={"ID":"ab738938-c8aa-4c7d-b230-6e0b385079d6","Type":"ContainerStarted","Data":"c8ed748154ac4e6753aba44b789e0114fa988727f24e1e672fbccb84c576f845"} Jul 06 19:50:01.036525 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:01.036479 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4ac0796b-7777-4b33-8a0e-a369697d8d88-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:50:01.039401 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:01.039267 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4ac0796b-7777-4b33-8a0e-a369697d8d88-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lkt7q\" (UID: \"4ac0796b-7777-4b33-8a0e-a369697d8d88\") " pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:50:01.065663 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:01.065627 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lkt7q" Jul 06 19:50:03.134198 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:03.134171 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lkt7q"] Jul 06 19:50:03.138477 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:50:03.138448 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ac0796b_7777_4b33_8a0e_a369697d8d88.slice/crio-9c790404bca07a36706f45f332e36d7ba78a96798aae502b92fcbbb6199f07b5 WatchSource:0}: Error finding container 9c790404bca07a36706f45f332e36d7ba78a96798aae502b92fcbbb6199f07b5: Status 404 returned error can't find the container with id 9c790404bca07a36706f45f332e36d7ba78a96798aae502b92fcbbb6199f07b5 Jul 06 19:50:03.520243 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:03.520208 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r9czf" event={"ID":"19b3e6d4-9980-4130-bff7-3a9f41085938","Type":"ContainerStarted","Data":"0d6ca82a5b10ccca1b9e70bb30a9c5ceb4c77f623830d63693190449f9c0a4b8"} Jul 06 19:50:03.520447 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:03.520259 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r9czf" event={"ID":"19b3e6d4-9980-4130-bff7-3a9f41085938","Type":"ContainerStarted","Data":"0a3b98c588ebaf95ac9daad561b610f905483bc4ca787046dbdfa376010e4429"} Jul 06 19:50:03.520447 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:03.520314 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-r9czf" Jul 06 19:50:03.521814 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:03.521778 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b564cbdf-6t5b9" event={"ID":"979fcd4b-bbbd-4ce5-b5fa-e6f6da95578d","Type":"ContainerStarted","Data":"ea6e3be7f29074def1d4b72d74e61e712920d9bfe22fec8611b2a9649d0e5502"} Jul 06 19:50:03.521921 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:03.521898 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b564cbdf-6t5b9" Jul 06 19:50:03.523234 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:03.523214 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-794xr" event={"ID":"ab738938-c8aa-4c7d-b230-6e0b385079d6","Type":"ContainerStarted","Data":"cf004e87d0f5dff91755cdd924ac71fb938d8e3d5105e0004ef6f7f9f435cdce"} Jul 06 19:50:03.523350 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:03.523339 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-794xr" Jul 06 19:50:03.524735 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:03.524200 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b564cbdf-6t5b9" Jul 06 19:50:03.525143 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:03.525126 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lkt7q" event={"ID":"4ac0796b-7777-4b33-8a0e-a369697d8d88","Type":"ContainerStarted","Data":"9c790404bca07a36706f45f332e36d7ba78a96798aae502b92fcbbb6199f07b5"} Jul 06 19:50:03.528083 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:03.528067 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-794xr" Jul 06 19:50:03.555214 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:03.555152 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-r9czf" podStartSLOduration=65.602088562 podStartE2EDuration="1m9.555134783s" totalImagesPullingTime="3.953046221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:03.55394377 +0000 UTC m=+103.001180489" watchObservedRunningTime="2026-07-06 19:50:03.555134783 +0000 UTC m=+103.002371499" Jul 06 19:50:03.585236 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:03.585117 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b564cbdf-6t5b9" podStartSLOduration=1.143649825 podStartE2EDuration="6.585097053s" totalImagesPullingTime="5.441447228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:49:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:03.584488307 +0000 UTC m=+103.031725043" watchObservedRunningTime="2026-07-06 19:50:03.585097053 +0000 UTC m=+103.032333749" Jul 06 19:50:03.606122 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:03.606059 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-794xr" podStartSLOduration=1.477868825 podStartE2EDuration="4.606041803s" totalImagesPullingTime="3.128172978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:49:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:03.605976084 +0000 UTC m=+103.053212803" watchObservedRunningTime="2026-07-06 19:50:03.606041803 +0000 UTC m=+103.053278520" Jul 06 19:50:07.541263 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:07.541225 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lkt7q" event={"ID":"4ac0796b-7777-4b33-8a0e-a369697d8d88","Type":"ContainerStarted","Data":"44232f07cb57dc8138e5842fa7a75ccbd223fb4599aa281c87e618da6f184b15"} Jul 06 19:50:08.546578 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:08.546541 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lkt7q" event={"ID":"4ac0796b-7777-4b33-8a0e-a369697d8d88","Type":"ContainerStarted","Data":"c0fc0056e32b9c20312e01dd3973976c98c5f243d38546bc04ae2c013d988ef9"} Jul 06 19:50:09.957605 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:09.957572 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-8wf5j"] Jul 06 19:50:09.962217 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:09.962185 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lssgd"] Jul 06 19:50:09.962362 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:09.962295 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8wf5j" Jul 06 19:50:09.964988 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:09.964965 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 19:50:09.965133 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:09.964998 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 06 19:50:09.965133 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:09.965040 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 19:50:09.965133 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:09.965052 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 06 19:50:09.965133 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:09.965000 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:09.966080 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:09.966056 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 19:50:09.968740 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:09.966508 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-7z9kd\"" Jul 06 19:50:09.968740 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:09.968308 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 19:50:09.968740 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:09.968499 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 19:50:09.968740 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:09.968561 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 19:50:09.968994 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:09.968792 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-s657x\"" Jul 06 19:50:09.975615 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:09.975592 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-8wf5j"] Jul 06 19:50:09.983040 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:09.983015 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55"] Jul 06 19:50:09.985914 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:09.985901 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" Jul 06 19:50:09.988571 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:09.988547 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 06 19:50:09.988571 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:09.988549 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 06 19:50:09.988777 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:09.988574 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-8zff8\"" Jul 06 19:50:09.988777 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:09.988587 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 06 19:50:09.995831 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:09.995813 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55"] Jul 06 19:50:10.089639 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.089608 2581 scope.go:117] "RemoveContainer" containerID="94d4e26e892f13ae8baaaaaa24925cddea651a7130f6f44e736e04045a676987" Jul 06 19:50:10.089817 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:50:10.089790 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-2xj4r_openshift-console-operator(25615b1c-46b5-484c-a198-88e9aac6fe21)\"" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" podUID="25615b1c-46b5-484c-a198-88e9aac6fe21" Jul 06 19:50:10.111146 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.111114 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b66e29b3-d6fd-4dd6-9110-5277755d2bba-node-exporter-textfile\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.111146 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.111160 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/100b14a8-13b5-48bc-a303-896ff7743bed-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-8wf5j\" (UID: \"100b14a8-13b5-48bc-a303-896ff7743bed\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8wf5j" Jul 06 19:50:10.111375 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.111179 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxpp7\" (UniqueName: \"kubernetes.io/projected/100b14a8-13b5-48bc-a303-896ff7743bed-kube-api-access-qxpp7\") pod \"openshift-state-metrics-54cf96dfcf-8wf5j\" (UID: \"100b14a8-13b5-48bc-a303-896ff7743bed\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8wf5j" Jul 06 19:50:10.111375 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.111271 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6dc6c9f-da74-41d5-80c4-08baf2959a41-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-cpt55\" (UID: \"c6dc6c9f-da74-41d5-80c4-08baf2959a41\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" Jul 06 19:50:10.111375 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.111313 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b66e29b3-d6fd-4dd6-9110-5277755d2bba-node-exporter-accelerators-collector-config\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.111516 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.111382 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/100b14a8-13b5-48bc-a303-896ff7743bed-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-8wf5j\" (UID: \"100b14a8-13b5-48bc-a303-896ff7743bed\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8wf5j" Jul 06 19:50:10.111516 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.111446 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk6dx\" (UniqueName: \"kubernetes.io/projected/c6dc6c9f-da74-41d5-80c4-08baf2959a41-kube-api-access-vk6dx\") pod \"kube-state-metrics-7c9dbb8b7d-cpt55\" (UID: \"c6dc6c9f-da74-41d5-80c4-08baf2959a41\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" Jul 06 19:50:10.111516 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.111480 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b66e29b3-d6fd-4dd6-9110-5277755d2bba-sys\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.111516 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.111508 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b66e29b3-d6fd-4dd6-9110-5277755d2bba-metrics-client-ca\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.111722 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.111584 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b66e29b3-d6fd-4dd6-9110-5277755d2bba-root\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.111722 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.111627 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c6dc6c9f-da74-41d5-80c4-08baf2959a41-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-cpt55\" (UID: \"c6dc6c9f-da74-41d5-80c4-08baf2959a41\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" Jul 06 19:50:10.111722 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.111660 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6dc6c9f-da74-41d5-80c4-08baf2959a41-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-cpt55\" (UID: \"c6dc6c9f-da74-41d5-80c4-08baf2959a41\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" Jul 06 19:50:10.111722 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.111688 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b66e29b3-d6fd-4dd6-9110-5277755d2bba-node-exporter-tls\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.111722 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.111717 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b66e29b3-d6fd-4dd6-9110-5277755d2bba-node-exporter-wtmp\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.111909 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.111749 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b66e29b3-d6fd-4dd6-9110-5277755d2bba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.111909 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.111790 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6dc6c9f-da74-41d5-80c4-08baf2959a41-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-cpt55\" (UID: \"c6dc6c9f-da74-41d5-80c4-08baf2959a41\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" Jul 06 19:50:10.111909 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.111816 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/100b14a8-13b5-48bc-a303-896ff7743bed-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-8wf5j\" (UID: \"100b14a8-13b5-48bc-a303-896ff7743bed\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8wf5j" Jul 06 19:50:10.111909 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.111856 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c6dc6c9f-da74-41d5-80c4-08baf2959a41-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-cpt55\" (UID: \"c6dc6c9f-da74-41d5-80c4-08baf2959a41\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" Jul 06 19:50:10.111909 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.111884 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s24zb\" (UniqueName: \"kubernetes.io/projected/b66e29b3-d6fd-4dd6-9110-5277755d2bba-kube-api-access-s24zb\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.212393 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.212301 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b66e29b3-d6fd-4dd6-9110-5277755d2bba-node-exporter-accelerators-collector-config\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.212393 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.212386 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/100b14a8-13b5-48bc-a303-896ff7743bed-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-8wf5j\" (UID: \"100b14a8-13b5-48bc-a303-896ff7743bed\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8wf5j" Jul 06 19:50:10.212624 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.212428 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk6dx\" (UniqueName: \"kubernetes.io/projected/c6dc6c9f-da74-41d5-80c4-08baf2959a41-kube-api-access-vk6dx\") pod \"kube-state-metrics-7c9dbb8b7d-cpt55\" (UID: \"c6dc6c9f-da74-41d5-80c4-08baf2959a41\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" Jul 06 19:50:10.212624 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.212536 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b66e29b3-d6fd-4dd6-9110-5277755d2bba-sys\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.212624 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.212573 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b66e29b3-d6fd-4dd6-9110-5277755d2bba-metrics-client-ca\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.212624 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.212605 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b66e29b3-d6fd-4dd6-9110-5277755d2bba-root\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.212818 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.212654 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c6dc6c9f-da74-41d5-80c4-08baf2959a41-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-cpt55\" (UID: \"c6dc6c9f-da74-41d5-80c4-08baf2959a41\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" Jul 06 19:50:10.212818 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.212703 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6dc6c9f-da74-41d5-80c4-08baf2959a41-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-cpt55\" (UID: \"c6dc6c9f-da74-41d5-80c4-08baf2959a41\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" Jul 06 19:50:10.212818 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.212732 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b66e29b3-d6fd-4dd6-9110-5277755d2bba-node-exporter-tls\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.212818 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.212762 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b66e29b3-d6fd-4dd6-9110-5277755d2bba-node-exporter-wtmp\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.212818 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.212769 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b66e29b3-d6fd-4dd6-9110-5277755d2bba-sys\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.212818 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.212795 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b66e29b3-d6fd-4dd6-9110-5277755d2bba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.213111 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.212828 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6dc6c9f-da74-41d5-80c4-08baf2959a41-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-cpt55\" (UID: \"c6dc6c9f-da74-41d5-80c4-08baf2959a41\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" Jul 06 19:50:10.213111 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.212859 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/100b14a8-13b5-48bc-a303-896ff7743bed-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-8wf5j\" (UID: \"100b14a8-13b5-48bc-a303-896ff7743bed\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8wf5j" Jul 06 19:50:10.213111 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.212890 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c6dc6c9f-da74-41d5-80c4-08baf2959a41-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-cpt55\" (UID: \"c6dc6c9f-da74-41d5-80c4-08baf2959a41\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" Jul 06 19:50:10.213111 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.212915 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s24zb\" (UniqueName: \"kubernetes.io/projected/b66e29b3-d6fd-4dd6-9110-5277755d2bba-kube-api-access-s24zb\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.213111 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.212972 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b66e29b3-d6fd-4dd6-9110-5277755d2bba-node-exporter-textfile\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.213111 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.212987 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b66e29b3-d6fd-4dd6-9110-5277755d2bba-node-exporter-accelerators-collector-config\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.213111 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.213005 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/100b14a8-13b5-48bc-a303-896ff7743bed-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-8wf5j\" (UID: \"100b14a8-13b5-48bc-a303-896ff7743bed\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8wf5j" Jul 06 19:50:10.213111 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.213029 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxpp7\" (UniqueName: \"kubernetes.io/projected/100b14a8-13b5-48bc-a303-896ff7743bed-kube-api-access-qxpp7\") pod \"openshift-state-metrics-54cf96dfcf-8wf5j\" (UID: \"100b14a8-13b5-48bc-a303-896ff7743bed\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8wf5j" Jul 06 19:50:10.213111 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:50:10.213084 2581 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 06 19:50:10.213585 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:50:10.213139 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6dc6c9f-da74-41d5-80c4-08baf2959a41-kube-state-metrics-tls podName:c6dc6c9f-da74-41d5-80c4-08baf2959a41 nodeName:}" failed. No retries permitted until 2026-07-06 19:50:10.713121248 +0000 UTC m=+110.160357963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/c6dc6c9f-da74-41d5-80c4-08baf2959a41-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-cpt55" (UID: "c6dc6c9f-da74-41d5-80c4-08baf2959a41") : secret "kube-state-metrics-tls" not found Jul 06 19:50:10.213585 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.213304 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b66e29b3-d6fd-4dd6-9110-5277755d2bba-metrics-client-ca\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.213585 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.213377 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b66e29b3-d6fd-4dd6-9110-5277755d2bba-node-exporter-textfile\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.213585 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.213423 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6dc6c9f-da74-41d5-80c4-08baf2959a41-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-cpt55\" (UID: \"c6dc6c9f-da74-41d5-80c4-08baf2959a41\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" Jul 06 19:50:10.213793 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.213667 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/c6dc6c9f-da74-41d5-80c4-08baf2959a41-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-cpt55\" (UID: \"c6dc6c9f-da74-41d5-80c4-08baf2959a41\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" Jul 06 19:50:10.213859 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.213812 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b66e29b3-d6fd-4dd6-9110-5277755d2bba-node-exporter-wtmp\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.213911 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.213855 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b66e29b3-d6fd-4dd6-9110-5277755d2bba-root\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.213962 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:50:10.213934 2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 19:50:10.214021 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:50:10.213982 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b66e29b3-d6fd-4dd6-9110-5277755d2bba-node-exporter-tls podName:b66e29b3-d6fd-4dd6-9110-5277755d2bba nodeName:}" failed. No retries permitted until 2026-07-06 19:50:10.7139661 +0000 UTC m=+110.161202797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b66e29b3-d6fd-4dd6-9110-5277755d2bba-node-exporter-tls") pod "node-exporter-lssgd" (UID: "b66e29b3-d6fd-4dd6-9110-5277755d2bba") : secret "node-exporter-tls" not found Jul 06 19:50:10.214075 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.214015 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/c6dc6c9f-da74-41d5-80c4-08baf2959a41-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-cpt55\" (UID: \"c6dc6c9f-da74-41d5-80c4-08baf2959a41\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" Jul 06 19:50:10.214075 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.214019 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/100b14a8-13b5-48bc-a303-896ff7743bed-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-8wf5j\" (UID: \"100b14a8-13b5-48bc-a303-896ff7743bed\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8wf5j" Jul 06 19:50:10.214597 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.214569 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6dc6c9f-da74-41d5-80c4-08baf2959a41-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-cpt55\" (UID: \"c6dc6c9f-da74-41d5-80c4-08baf2959a41\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" Jul 06 19:50:10.216298 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.216271 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b66e29b3-d6fd-4dd6-9110-5277755d2bba-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.216494 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.216476 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6dc6c9f-da74-41d5-80c4-08baf2959a41-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-cpt55\" (UID: \"c6dc6c9f-da74-41d5-80c4-08baf2959a41\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" Jul 06 19:50:10.216740 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.216716 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/100b14a8-13b5-48bc-a303-896ff7743bed-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-8wf5j\" (UID: \"100b14a8-13b5-48bc-a303-896ff7743bed\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8wf5j" Jul 06 19:50:10.216786 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.216764 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/100b14a8-13b5-48bc-a303-896ff7743bed-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-8wf5j\" (UID: \"100b14a8-13b5-48bc-a303-896ff7743bed\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8wf5j" Jul 06 19:50:10.223862 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.223835 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk6dx\" (UniqueName: \"kubernetes.io/projected/c6dc6c9f-da74-41d5-80c4-08baf2959a41-kube-api-access-vk6dx\") pod \"kube-state-metrics-7c9dbb8b7d-cpt55\" (UID: \"c6dc6c9f-da74-41d5-80c4-08baf2959a41\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" Jul 06 19:50:10.224052 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.224031 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxpp7\" (UniqueName: \"kubernetes.io/projected/100b14a8-13b5-48bc-a303-896ff7743bed-kube-api-access-qxpp7\") pod \"openshift-state-metrics-54cf96dfcf-8wf5j\" (UID: \"100b14a8-13b5-48bc-a303-896ff7743bed\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8wf5j" Jul 06 19:50:10.224284 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.224261 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s24zb\" (UniqueName: \"kubernetes.io/projected/b66e29b3-d6fd-4dd6-9110-5277755d2bba-kube-api-access-s24zb\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.276403 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.276365 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8wf5j" Jul 06 19:50:10.718279 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.718248 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b66e29b3-d6fd-4dd6-9110-5277755d2bba-node-exporter-tls\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.718442 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.718305 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6dc6c9f-da74-41d5-80c4-08baf2959a41-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-cpt55\" (UID: \"c6dc6c9f-da74-41d5-80c4-08baf2959a41\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" Jul 06 19:50:10.720486 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.720459 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/c6dc6c9f-da74-41d5-80c4-08baf2959a41-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-cpt55\" (UID: \"c6dc6c9f-da74-41d5-80c4-08baf2959a41\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" Jul 06 19:50:10.720587 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.720465 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b66e29b3-d6fd-4dd6-9110-5277755d2bba-node-exporter-tls\") pod \"node-exporter-lssgd\" (UID: \"b66e29b3-d6fd-4dd6-9110-5277755d2bba\") " pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.748636 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.748605 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-8wf5j"] Jul 06 19:50:10.751146 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:50:10.751115 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod100b14a8_13b5_48bc_a303_896ff7743bed.slice/crio-e5510bc2dee8c8af660f0a13f37f8287bd9d23015809407b02fdbf80a34fae53 WatchSource:0}: Error finding container e5510bc2dee8c8af660f0a13f37f8287bd9d23015809407b02fdbf80a34fae53: Status 404 returned error can't find the container with id e5510bc2dee8c8af660f0a13f37f8287bd9d23015809407b02fdbf80a34fae53 Jul 06 19:50:10.884472 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.884396 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lssgd" Jul 06 19:50:10.893708 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:50:10.893680 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb66e29b3_d6fd_4dd6_9110_5277755d2bba.slice/crio-b8d30f86fd8cabd7f944f64445e274414f08db6166bbf26f2e0d8a1d29b326f6 WatchSource:0}: Error finding container b8d30f86fd8cabd7f944f64445e274414f08db6166bbf26f2e0d8a1d29b326f6: Status 404 returned error can't find the container with id b8d30f86fd8cabd7f944f64445e274414f08db6166bbf26f2e0d8a1d29b326f6 Jul 06 19:50:10.895018 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:10.894970 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" Jul 06 19:50:11.038617 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.038595 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55"] Jul 06 19:50:11.040776 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:50:11.040750 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6dc6c9f_da74_41d5_80c4_08baf2959a41.slice/crio-db161072a8e43beb19844b0d612e8dcd9d8a777bf08dbb4f3a2f8b72b1d452dc WatchSource:0}: Error finding container db161072a8e43beb19844b0d612e8dcd9d8a777bf08dbb4f3a2f8b72b1d452dc: Status 404 returned error can't find the container with id db161072a8e43beb19844b0d612e8dcd9d8a777bf08dbb4f3a2f8b72b1d452dc Jul 06 19:50:11.054201 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.054179 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:50:11.058393 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.058376 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.061360 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.061198 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 19:50:11.061360 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.061217 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 19:50:11.061360 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.061234 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 19:50:11.061360 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.061229 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 19:50:11.061360 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.061270 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 19:50:11.061360 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.061291 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 19:50:11.061711 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.061293 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-vzkkh\"" Jul 06 19:50:11.061711 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.061381 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 19:50:11.061812 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.061731 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 19:50:11.061812 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.061768 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 19:50:11.074756 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.074722 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:50:11.221893 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.221796 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.221893 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.221840 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.221893 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.221873 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-config-out\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.222144 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.221930 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.222144 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.222035 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-web-config\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.222144 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.222079 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.222144 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.222127 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9bd2\" (UniqueName: \"kubernetes.io/projected/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-kube-api-access-z9bd2\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.222372 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.222154 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.222372 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.222208 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.222372 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.222257 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-config-volume\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.222372 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.222288 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.222372 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.222358 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.222546 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.222383 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.322902 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.322867 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.323103 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.322914 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-config-out\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.323103 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.322930 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.323103 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.322948 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-web-config\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.323103 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.322964 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.323103 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.322987 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9bd2\" (UniqueName: \"kubernetes.io/projected/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-kube-api-access-z9bd2\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.323103 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.323015 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.323103 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.323046 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.323103 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.323080 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-config-volume\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.323103 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.323105 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.323593 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.323142 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.323593 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.323161 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.323593 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.323197 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.323593 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.323493 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.324344 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.324293 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.325779 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.325747 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-config-out\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.325884 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.325822 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.326054 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.326028 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.326113 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.326068 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.326452 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.326432 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.326722 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.326694 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.326795 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.326708 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.327608 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.327589 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-web-config\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.327931 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.327909 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.328036 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.328014 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-config-volume\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.330781 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.330757 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9bd2\" (UniqueName: \"kubernetes.io/projected/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-kube-api-access-z9bd2\") pod \"alertmanager-main-0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.367386 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.367348 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:50:11.506784 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.506690 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:50:11.509140 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:50:11.509117 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0eaec17_213b_4d1e_8aa8_7fe5013997e0.slice/crio-584cb714ac21743f5a761850f1911f06221127aacfbfefba9f97ddaea01a00b0 WatchSource:0}: Error finding container 584cb714ac21743f5a761850f1911f06221127aacfbfefba9f97ddaea01a00b0: Status 404 returned error can't find the container with id 584cb714ac21743f5a761850f1911f06221127aacfbfefba9f97ddaea01a00b0 Jul 06 19:50:11.559430 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.559393 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lkt7q" event={"ID":"4ac0796b-7777-4b33-8a0e-a369697d8d88","Type":"ContainerStarted","Data":"b919d0d3d429e3b26c64342f8fab43cbb23c945f0e7c4dbe2736baa022cab1eb"} Jul 06 19:50:11.560531 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.560502 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lssgd" event={"ID":"b66e29b3-d6fd-4dd6-9110-5277755d2bba","Type":"ContainerStarted","Data":"b8d30f86fd8cabd7f944f64445e274414f08db6166bbf26f2e0d8a1d29b326f6"} Jul 06 19:50:11.562096 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.562075 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8wf5j" event={"ID":"100b14a8-13b5-48bc-a303-896ff7743bed","Type":"ContainerStarted","Data":"f0cb67712740e8eea6ef928b03684c6a18e3115231f1b611ff87deaf9ff1dd40"} Jul 06 19:50:11.562215 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.562113 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8wf5j" event={"ID":"100b14a8-13b5-48bc-a303-896ff7743bed","Type":"ContainerStarted","Data":"ecaba7e205932054827f1f7e40e855d04357c6065b063256999ca077cce0bf57"} Jul 06 19:50:11.562215 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.562140 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8wf5j" event={"ID":"100b14a8-13b5-48bc-a303-896ff7743bed","Type":"ContainerStarted","Data":"e5510bc2dee8c8af660f0a13f37f8287bd9d23015809407b02fdbf80a34fae53"} Jul 06 19:50:11.564045 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.564009 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0eaec17-213b-4d1e-8aa8-7fe5013997e0","Type":"ContainerStarted","Data":"584cb714ac21743f5a761850f1911f06221127aacfbfefba9f97ddaea01a00b0"} Jul 06 19:50:11.565220 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.565201 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" event={"ID":"c6dc6c9f-da74-41d5-80c4-08baf2959a41","Type":"ContainerStarted","Data":"db161072a8e43beb19844b0d612e8dcd9d8a777bf08dbb4f3a2f8b72b1d452dc"} Jul 06 19:50:11.579004 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:11.578964 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lkt7q" podStartSLOduration=35.172799581 podStartE2EDuration="42.578946914s" totalImagesPullingTime="7.406147333s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:49:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:11.578093229 +0000 UTC m=+111.025329949" watchObservedRunningTime="2026-07-06 19:50:11.578946914 +0000 UTC m=+111.026183633" Jul 06 19:50:13.531435 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:13.531411 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-r9czf" Jul 06 19:50:13.575093 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:13.575065 2581 generic.go:356] "Generic (PLEG): container finished" podID="b66e29b3-d6fd-4dd6-9110-5277755d2bba" containerID="fe6de43947040de1bc7870bf8d862102dc933bcd5057382ebaeb104307b37132" exitCode=0 Jul 06 19:50:13.575260 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:13.575152 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lssgd" event={"ID":"b66e29b3-d6fd-4dd6-9110-5277755d2bba","Type":"ContainerDied","Data":"fe6de43947040de1bc7870bf8d862102dc933bcd5057382ebaeb104307b37132"} Jul 06 19:50:13.577442 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:13.577412 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8wf5j" event={"ID":"100b14a8-13b5-48bc-a303-896ff7743bed","Type":"ContainerStarted","Data":"381e3b7f8478373a89869e61c1e5ef8ebde0e1ad22a0b0f82e244c3a7a257c6c"} Jul 06 19:50:13.579027 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:13.579003 2581 generic.go:356] "Generic (PLEG): container finished" podID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerID="e07a84f8b904091680781683bc33f8f6d11849f9e2c6634f563704392347b03c" exitCode=0 Jul 06 19:50:13.579134 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:13.579089 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0eaec17-213b-4d1e-8aa8-7fe5013997e0","Type":"ContainerDied","Data":"e07a84f8b904091680781683bc33f8f6d11849f9e2c6634f563704392347b03c"} Jul 06 19:50:13.581655 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:13.581633 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" event={"ID":"c6dc6c9f-da74-41d5-80c4-08baf2959a41","Type":"ContainerStarted","Data":"8bd1b7ea157823141981bfcd384b9da4f0cf020eb2c5b076581be8ca7db794d7"} Jul 06 19:50:13.581739 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:13.581683 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" event={"ID":"c6dc6c9f-da74-41d5-80c4-08baf2959a41","Type":"ContainerStarted","Data":"f1e1a84b672550ac92436671b124dd887b9ad6d3e6841ddc6bc587bbe4e1f4cb"} Jul 06 19:50:13.581739 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:13.581715 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" event={"ID":"c6dc6c9f-da74-41d5-80c4-08baf2959a41","Type":"ContainerStarted","Data":"7dc593bf1f560e5a8f05fa1bf9c5a35a0c7b894b25351ba3899571ba710c5307"} Jul 06 19:50:13.642575 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:13.642522 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-8wf5j" podStartSLOduration=2.841042581 podStartE2EDuration="4.64250525s" totalImagesPullingTime="1.801462669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:50:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:13.641766686 +0000 UTC m=+113.089003405" watchObservedRunningTime="2026-07-06 19:50:13.64250525 +0000 UTC m=+113.089741968" Jul 06 19:50:13.663517 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:13.663471 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-cpt55" podStartSLOduration=3.050329221 podStartE2EDuration="4.663454283s" totalImagesPullingTime="1.613125062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:50:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:13.662853577 +0000 UTC m=+113.110090297" watchObservedRunningTime="2026-07-06 19:50:13.663454283 +0000 UTC m=+113.110691000" Jul 06 19:50:14.589215 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:14.589175 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lssgd" event={"ID":"b66e29b3-d6fd-4dd6-9110-5277755d2bba","Type":"ContainerStarted","Data":"db6894c6fb51e66c96082744a1cf8028571cafe1cd93cf772c8bf153fcf5becb"} Jul 06 19:50:14.589672 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:14.589246 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lssgd" event={"ID":"b66e29b3-d6fd-4dd6-9110-5277755d2bba","Type":"ContainerStarted","Data":"fc36322eac97f759d0c67a7fa2353df33ee9b8e4e07b72fbdd6889a3cb57a121"} Jul 06 19:50:14.610888 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:14.610835 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lssgd" podStartSLOduration=3.772571235 podStartE2EDuration="5.610816107s" totalImagesPullingTime="1.758057308s" totalInitContainerRuntime="80.187564ms" isStatefulPod=true podCreationTimestamp="2026-07-06 19:50:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:14.610218814 +0000 UTC m=+114.057455549" watchObservedRunningTime="2026-07-06 19:50:14.610816107 +0000 UTC m=+114.058052830" Jul 06 19:50:14.737196 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:14.737163 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-nfrhc"] Jul 06 19:50:14.740220 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:14.740202 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nfrhc" Jul 06 19:50:14.742664 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:14.742638 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 06 19:50:14.742782 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:14.742638 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-c8ms6\"" Jul 06 19:50:14.749804 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:14.749782 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-nfrhc"] Jul 06 19:50:14.860610 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:14.860517 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/85fe5aa1-7407-4735-aede-ef94e7bb4800-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-nfrhc\" (UID: \"85fe5aa1-7407-4735-aede-ef94e7bb4800\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nfrhc" Jul 06 19:50:14.860741 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:14.860671 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m52fk\" (UniqueName: \"kubernetes.io/projected/85fe5aa1-7407-4735-aede-ef94e7bb4800-kube-api-access-m52fk\") pod \"monitoring-plugin-d5dcff7cf-nfrhc\" (UID: \"85fe5aa1-7407-4735-aede-ef94e7bb4800\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nfrhc" Jul 06 19:50:14.961091 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:14.961050 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m52fk\" (UniqueName: \"kubernetes.io/projected/85fe5aa1-7407-4735-aede-ef94e7bb4800-kube-api-access-m52fk\") pod \"monitoring-plugin-d5dcff7cf-nfrhc\" (UID: \"85fe5aa1-7407-4735-aede-ef94e7bb4800\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nfrhc" Jul 06 19:50:14.961273 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:14.961127 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/85fe5aa1-7407-4735-aede-ef94e7bb4800-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-nfrhc\" (UID: \"85fe5aa1-7407-4735-aede-ef94e7bb4800\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nfrhc" Jul 06 19:50:14.963849 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:14.963819 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/85fe5aa1-7407-4735-aede-ef94e7bb4800-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-nfrhc\" (UID: \"85fe5aa1-7407-4735-aede-ef94e7bb4800\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nfrhc" Jul 06 19:50:14.969635 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:14.969606 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m52fk\" (UniqueName: \"kubernetes.io/projected/85fe5aa1-7407-4735-aede-ef94e7bb4800-kube-api-access-m52fk\") pod \"monitoring-plugin-d5dcff7cf-nfrhc\" (UID: \"85fe5aa1-7407-4735-aede-ef94e7bb4800\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nfrhc" Jul 06 19:50:15.049314 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.049266 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nfrhc" Jul 06 19:50:15.162479 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.162441 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh"] Jul 06 19:50:15.167214 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.167190 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.170926 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.170906 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 06 19:50:15.171538 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.171520 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 06 19:50:15.171680 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.171655 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 06 19:50:15.171755 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.171740 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 06 19:50:15.171811 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.171764 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-pfc9g\"" Jul 06 19:50:15.171974 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.171954 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 06 19:50:15.178197 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.178165 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 06 19:50:15.179368 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.179347 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-nfrhc"] Jul 06 19:50:15.182398 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:50:15.182370 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85fe5aa1_7407_4735_aede_ef94e7bb4800.slice/crio-a63cd6154b9fcdf9c03bb65ce77a2beaef5b1b6df4656f9995233c5637ef576a WatchSource:0}: Error finding container a63cd6154b9fcdf9c03bb65ce77a2beaef5b1b6df4656f9995233c5637ef576a: Status 404 returned error can't find the container with id a63cd6154b9fcdf9c03bb65ce77a2beaef5b1b6df4656f9995233c5637ef576a Jul 06 19:50:15.183117 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.183095 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh"] Jul 06 19:50:15.263691 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.263655 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22715c7b-7254-4d6c-a760-2c58e4f071a7-metrics-client-ca\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.263847 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.263722 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/22715c7b-7254-4d6c-a760-2c58e4f071a7-telemeter-client-tls\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.263847 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.263819 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgxv4\" (UniqueName: \"kubernetes.io/projected/22715c7b-7254-4d6c-a760-2c58e4f071a7-kube-api-access-xgxv4\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.263922 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.263852 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/22715c7b-7254-4d6c-a760-2c58e4f071a7-secret-telemeter-client\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.263922 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.263879 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22715c7b-7254-4d6c-a760-2c58e4f071a7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.263922 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.263912 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22715c7b-7254-4d6c-a760-2c58e4f071a7-serving-certs-ca-bundle\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.264011 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.263933 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/22715c7b-7254-4d6c-a760-2c58e4f071a7-federate-client-tls\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.264011 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.263961 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22715c7b-7254-4d6c-a760-2c58e4f071a7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.364988 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.364955 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/22715c7b-7254-4d6c-a760-2c58e4f071a7-telemeter-client-tls\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.365171 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.365002 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgxv4\" (UniqueName: \"kubernetes.io/projected/22715c7b-7254-4d6c-a760-2c58e4f071a7-kube-api-access-xgxv4\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.365171 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.365027 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/22715c7b-7254-4d6c-a760-2c58e4f071a7-secret-telemeter-client\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.365171 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.365048 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22715c7b-7254-4d6c-a760-2c58e4f071a7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.365171 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.365065 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22715c7b-7254-4d6c-a760-2c58e4f071a7-serving-certs-ca-bundle\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.365171 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.365084 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/22715c7b-7254-4d6c-a760-2c58e4f071a7-federate-client-tls\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.365171 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.365116 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22715c7b-7254-4d6c-a760-2c58e4f071a7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.365171 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.365142 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22715c7b-7254-4d6c-a760-2c58e4f071a7-metrics-client-ca\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.365921 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.365889 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22715c7b-7254-4d6c-a760-2c58e4f071a7-serving-certs-ca-bundle\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.366025 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.365903 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/22715c7b-7254-4d6c-a760-2c58e4f071a7-metrics-client-ca\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.366178 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.366157 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/22715c7b-7254-4d6c-a760-2c58e4f071a7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.367835 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.367804 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/22715c7b-7254-4d6c-a760-2c58e4f071a7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.367923 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.367886 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/22715c7b-7254-4d6c-a760-2c58e4f071a7-telemeter-client-tls\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.368100 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.368077 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/22715c7b-7254-4d6c-a760-2c58e4f071a7-federate-client-tls\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.368144 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.368120 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/22715c7b-7254-4d6c-a760-2c58e4f071a7-secret-telemeter-client\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.373176 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.373151 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgxv4\" (UniqueName: \"kubernetes.io/projected/22715c7b-7254-4d6c-a760-2c58e4f071a7-kube-api-access-xgxv4\") pod \"telemeter-client-54f8cd97b5-sjsdh\" (UID: \"22715c7b-7254-4d6c-a760-2c58e4f071a7\") " pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.477670 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.477587 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" Jul 06 19:50:15.595592 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.595546 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nfrhc" event={"ID":"85fe5aa1-7407-4735-aede-ef94e7bb4800","Type":"ContainerStarted","Data":"a63cd6154b9fcdf9c03bb65ce77a2beaef5b1b6df4656f9995233c5637ef576a"} Jul 06 19:50:15.625561 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:15.625528 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh"] Jul 06 19:50:15.851746 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:50:15.851704 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22715c7b_7254_4d6c_a760_2c58e4f071a7.slice/crio-d428f0818ed62f4a3b322596a7b68491f8c19c3a92a14edd6c564abfd8238be8 WatchSource:0}: Error finding container d428f0818ed62f4a3b322596a7b68491f8c19c3a92a14edd6c564abfd8238be8: Status 404 returned error can't find the container with id d428f0818ed62f4a3b322596a7b68491f8c19c3a92a14edd6c564abfd8238be8 Jul 06 19:50:16.186242 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.185922 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:50:16.191797 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.191773 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.195637 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.195609 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 19:50:16.195733 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.195652 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 19:50:16.195733 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.195678 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 19:50:16.195931 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.195911 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 19:50:16.196135 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.196118 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6qtbb\"" Jul 06 19:50:16.198212 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.196840 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 19:50:16.198212 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.197620 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 19:50:16.198212 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.198065 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 19:50:16.198875 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.198801 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 19:50:16.198987 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.198913 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 19:50:16.200257 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.199096 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 19:50:16.200257 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.199261 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 19:50:16.200257 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.199428 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 19:50:16.200257 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.199558 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-84gg8b8u1nktj\"" Jul 06 19:50:16.200257 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.199755 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 19:50:16.200906 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.200719 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 19:50:16.202452 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.202010 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 19:50:16.212459 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.212433 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:50:16.374993 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.374903 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.374993 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.374950 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.374993 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.374967 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.374993 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.374985 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/458a7e6e-c625-401c-a9b2-75ea36214fe5-config-out\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.375245 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.375028 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.375245 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.375087 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/458a7e6e-c625-401c-a9b2-75ea36214fe5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.375245 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.375119 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.375245 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.375161 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.375245 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.375211 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-web-config\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.375447 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.375255 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8zvc\" (UniqueName: \"kubernetes.io/projected/458a7e6e-c625-401c-a9b2-75ea36214fe5-kube-api-access-t8zvc\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.375447 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.375272 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-config\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.375447 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.375292 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.375447 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.375310 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.375447 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.375353 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.375447 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.375373 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.375447 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.375408 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.375447 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.375430 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.375697 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.375466 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.375697 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.375492 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.375697 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.375522 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.476468 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.476435 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.476468 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.476471 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.476715 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.476496 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/458a7e6e-c625-401c-a9b2-75ea36214fe5-config-out\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.477092 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.476915 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.477092 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.476985 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/458a7e6e-c625-401c-a9b2-75ea36214fe5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.477092 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.477032 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.477092 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.477056 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.477092 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.477080 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-web-config\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.477483 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.477130 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8zvc\" (UniqueName: \"kubernetes.io/projected/458a7e6e-c625-401c-a9b2-75ea36214fe5-kube-api-access-t8zvc\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.477483 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.477157 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-config\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.477483 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.477184 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.477483 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.477209 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.477483 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.477237 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.477483 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.477259 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.477483 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.477294 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.477483 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.477342 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.477483 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.477409 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.477483 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.477433 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.477483 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.477461 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.478010 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.477502 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.480266 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.478777 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.480266 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.479483 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/458a7e6e-c625-401c-a9b2-75ea36214fe5-config-out\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.480266 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.479698 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.480266 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.479795 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.480266 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.480116 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/458a7e6e-c625-401c-a9b2-75ea36214fe5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.480266 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.480178 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.480774 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.480724 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.480891 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.480803 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.481763 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.481201 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.481763 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.481208 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.481763 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.481268 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.481763 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.481725 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.484555 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.482627 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.484555 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.483527 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.484555 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.483834 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-web-config\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.484555 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.483958 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-config\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.484555 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.483969 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.484555 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.484153 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.484906 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.484888 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.488913 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.488892 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8zvc\" (UniqueName: \"kubernetes.io/projected/458a7e6e-c625-401c-a9b2-75ea36214fe5-kube-api-access-t8zvc\") pod \"prometheus-k8s-0\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.518503 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.518474 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:16.600723 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.600683 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" event={"ID":"22715c7b-7254-4d6c-a760-2c58e4f071a7","Type":"ContainerStarted","Data":"d428f0818ed62f4a3b322596a7b68491f8c19c3a92a14edd6c564abfd8238be8"} Jul 06 19:50:16.604278 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.604247 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0eaec17-213b-4d1e-8aa8-7fe5013997e0","Type":"ContainerStarted","Data":"420b942943c397d1cd081282c500931770ec375f2d84d0d739228c10e8a3e2b4"} Jul 06 19:50:16.604444 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.604349 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0eaec17-213b-4d1e-8aa8-7fe5013997e0","Type":"ContainerStarted","Data":"8f1936f4195b2847ab0ef52d23f72c33a7af355954aeb457a768e2c1225b0228"} Jul 06 19:50:16.604444 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.604408 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0eaec17-213b-4d1e-8aa8-7fe5013997e0","Type":"ContainerStarted","Data":"f62ac8465eb0bb1dc546d17603abf4b704c74f6633796d54ba54d124dec31dd5"} Jul 06 19:50:16.604558 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.604454 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0eaec17-213b-4d1e-8aa8-7fe5013997e0","Type":"ContainerStarted","Data":"c413d5e8e78ef3e53bf6c2a1553bec611d407f46d98b3de551979a05f572dc78"} Jul 06 19:50:16.604558 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:16.604493 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0eaec17-213b-4d1e-8aa8-7fe5013997e0","Type":"ContainerStarted","Data":"ca3626efd58d37c1801379a7d596ce7ccbc8ff2a1d4b1896673a70f4aaa59282"} Jul 06 19:50:17.029272 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:17.029250 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:50:17.031945 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:50:17.031920 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod458a7e6e_c625_401c_a9b2_75ea36214fe5.slice/crio-36cccac5d8ba0acc0a94f03e9b20a679744e7d16211e08f6de243e70f3bf4936 WatchSource:0}: Error finding container 36cccac5d8ba0acc0a94f03e9b20a679744e7d16211e08f6de243e70f3bf4936: Status 404 returned error can't find the container with id 36cccac5d8ba0acc0a94f03e9b20a679744e7d16211e08f6de243e70f3bf4936 Jul 06 19:50:17.609918 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:17.609859 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nfrhc" event={"ID":"85fe5aa1-7407-4735-aede-ef94e7bb4800","Type":"ContainerStarted","Data":"c575ec71d11cd020d92dce858c2ae65b45ab8722287e770e05b02cf4fb8ffe8f"} Jul 06 19:50:17.610411 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:17.610008 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nfrhc" Jul 06 19:50:17.614521 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:17.614447 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0eaec17-213b-4d1e-8aa8-7fe5013997e0","Type":"ContainerStarted","Data":"4d17dad36ffd917f9de302d9a294af52e099bbd91c7d7055f95aae841bee7c78"} Jul 06 19:50:17.615926 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:17.615890 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nfrhc" Jul 06 19:50:17.616418 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:17.616395 2581 generic.go:356] "Generic (PLEG): container finished" podID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerID="1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f" exitCode=0 Jul 06 19:50:17.616500 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:17.616436 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"458a7e6e-c625-401c-a9b2-75ea36214fe5","Type":"ContainerDied","Data":"1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f"} Jul 06 19:50:17.616542 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:17.616496 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"458a7e6e-c625-401c-a9b2-75ea36214fe5","Type":"ContainerStarted","Data":"36cccac5d8ba0acc0a94f03e9b20a679744e7d16211e08f6de243e70f3bf4936"} Jul 06 19:50:17.633553 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:17.633503 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-nfrhc" podStartSLOduration=1.900392769 podStartE2EDuration="3.633485423s" totalImagesPullingTime="1.733092654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:50:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:17.632528573 +0000 UTC m=+117.079765291" watchObservedRunningTime="2026-07-06 19:50:17.633485423 +0000 UTC m=+117.080722141" Jul 06 19:50:17.687499 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:17.687446 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.431085943 podStartE2EDuration="6.687431279s" totalImagesPullingTime="5.240935153s" totalInitContainerRuntime="15.410183ms" isStatefulPod=false podCreationTimestamp="2026-07-06 19:50:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:17.685284618 +0000 UTC m=+117.132521336" watchObservedRunningTime="2026-07-06 19:50:17.687431279 +0000 UTC m=+117.134667997" Jul 06 19:50:18.623906 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:18.623784 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" event={"ID":"22715c7b-7254-4d6c-a760-2c58e4f071a7","Type":"ContainerStarted","Data":"43ed72bf6c195846d4f016c9ec89dee0293775b2d2480aa344b21e539d283a54"} Jul 06 19:50:18.623906 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:18.623866 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" event={"ID":"22715c7b-7254-4d6c-a760-2c58e4f071a7","Type":"ContainerStarted","Data":"c01c84d27f899a4159618b8e6f8e6b1fa89522b2183bd103a834a82b550244f5"} Jul 06 19:50:18.836742 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:18.836706 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65ddd84d6b-j244t"] Jul 06 19:50:18.841201 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:18.841158 2581 patch_prober.go:28] interesting pod/image-registry-65ddd84d6b-j244t container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 06 19:50:18.841343 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:18.841217 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" podUID="121e68c7-16d6-4c5a-a36b-9149e3561efc" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 19:50:19.634490 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:19.634442 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" event={"ID":"22715c7b-7254-4d6c-a760-2c58e4f071a7","Type":"ContainerStarted","Data":"db2ee57368a24ef2923df9f59a648f8f4110130b764b218ac81b8324492707b8"} Jul 06 19:50:19.663163 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:19.663109 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-54f8cd97b5-sjsdh" podStartSLOduration=2.078833307 podStartE2EDuration="4.663091423s" totalImagesPullingTime="2.584258116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:50:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:19.661303899 +0000 UTC m=+119.108540618" watchObservedRunningTime="2026-07-06 19:50:19.663091423 +0000 UTC m=+119.110328139" Jul 06 19:50:21.647291 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:21.647257 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"458a7e6e-c625-401c-a9b2-75ea36214fe5","Type":"ContainerStarted","Data":"af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e"} Jul 06 19:50:22.089641 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:22.089611 2581 scope.go:117] "RemoveContainer" containerID="94d4e26e892f13ae8baaaaaa24925cddea651a7130f6f44e736e04045a676987" Jul 06 19:50:22.653962 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:22.653912 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/2.log" Jul 06 19:50:22.654464 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:22.654035 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" event={"ID":"25615b1c-46b5-484c-a198-88e9aac6fe21","Type":"ContainerStarted","Data":"521663eb345c95990fb08c30e69d3a3c8a44d9ebe9e946e17ee6dced94ee1f2a"} Jul 06 19:50:22.654464 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:22.654379 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" Jul 06 19:50:22.656753 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:22.656721 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"458a7e6e-c625-401c-a9b2-75ea36214fe5","Type":"ContainerStarted","Data":"467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb"} Jul 06 19:50:22.846963 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:22.846929 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-2xj4r" Jul 06 19:50:24.667390 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:24.667351 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"458a7e6e-c625-401c-a9b2-75ea36214fe5","Type":"ContainerStarted","Data":"254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f"} Jul 06 19:50:24.667801 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:24.667431 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"458a7e6e-c625-401c-a9b2-75ea36214fe5","Type":"ContainerStarted","Data":"f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76"} Jul 06 19:50:24.667801 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:24.667470 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"458a7e6e-c625-401c-a9b2-75ea36214fe5","Type":"ContainerStarted","Data":"b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088"} Jul 06 19:50:24.667801 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:24.667505 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"458a7e6e-c625-401c-a9b2-75ea36214fe5","Type":"ContainerStarted","Data":"bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c"} Jul 06 19:50:24.702719 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:24.702657 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.719086096 podStartE2EDuration="8.702640998s" totalImagesPullingTime="5.962988531s" totalInitContainerRuntime="20.566371ms" isStatefulPod=false podCreationTimestamp="2026-07-06 19:50:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:24.700697231 +0000 UTC m=+124.147933960" watchObservedRunningTime="2026-07-06 19:50:24.702640998 +0000 UTC m=+124.149877716" Jul 06 19:50:26.519058 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:26.519021 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:50:28.841336 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:28.841290 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:50:29.906808 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:29.906771 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs\") pod \"network-metrics-daemon-tgl28\" (UID: \"2df21624-1cb1-4437-99a7-b91b4784d32d\") " pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:50:29.909026 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:29.909005 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2df21624-1cb1-4437-99a7-b91b4784d32d-metrics-certs\") pod \"network-metrics-daemon-tgl28\" (UID: \"2df21624-1cb1-4437-99a7-b91b4784d32d\") " pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:50:30.201687 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:30.201593 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9xsp7\"" Jul 06 19:50:30.209386 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:30.209352 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tgl28" Jul 06 19:50:30.371657 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:30.371626 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tgl28"] Jul 06 19:50:30.381023 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:50:30.380980 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2df21624_1cb1_4437_99a7_b91b4784d32d.slice/crio-f21b2741122364379ec43fc578073a392df369ce0b92c19a325c0b36ffbfe595 WatchSource:0}: Error finding container f21b2741122364379ec43fc578073a392df369ce0b92c19a325c0b36ffbfe595: Status 404 returned error can't find the container with id f21b2741122364379ec43fc578073a392df369ce0b92c19a325c0b36ffbfe595 Jul 06 19:50:30.695452 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:30.695421 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tgl28" event={"ID":"2df21624-1cb1-4437-99a7-b91b4784d32d","Type":"ContainerStarted","Data":"f21b2741122364379ec43fc578073a392df369ce0b92c19a325c0b36ffbfe595"} Jul 06 19:50:31.701421 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:31.701381 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tgl28" event={"ID":"2df21624-1cb1-4437-99a7-b91b4784d32d","Type":"ContainerStarted","Data":"668d4b3b392ba40a71c3190d1b708f33063164eccafbbfdcb23047a23a714662"} Jul 06 19:50:32.707597 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:32.707562 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tgl28" event={"ID":"2df21624-1cb1-4437-99a7-b91b4784d32d","Type":"ContainerStarted","Data":"d72fbf32615fcac063c7a6f74b7ed5e6f1a2c6c54c3173fcc5c468f32b75e971"} Jul 06 19:50:32.730875 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:32.730804 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-tgl28" podStartSLOduration=130.577385305 podStartE2EDuration="2m11.730790961s" totalImagesPullingTime="1.153405656s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:48:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:50:32.729538995 +0000 UTC m=+132.176775714" watchObservedRunningTime="2026-07-06 19:50:32.730790961 +0000 UTC m=+132.178027679" Jul 06 19:50:43.857093 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:43.857015 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" podUID="121e68c7-16d6-4c5a-a36b-9149e3561efc" containerName="registry" containerID="cri-o://d2048ff618a49c90da922ef8988e952136af12fc0dec07aedfb21c66cbe2eca4" gracePeriod=30 Jul 06 19:50:44.126247 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.126173 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:50:44.230745 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.230711 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-certificates\") pod \"121e68c7-16d6-4c5a-a36b-9149e3561efc\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " Jul 06 19:50:44.230745 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.230742 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjpjg\" (UniqueName: \"kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-kube-api-access-zjpjg\") pod \"121e68c7-16d6-4c5a-a36b-9149e3561efc\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " Jul 06 19:50:44.230981 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.230773 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls\") pod \"121e68c7-16d6-4c5a-a36b-9149e3561efc\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " Jul 06 19:50:44.230981 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.230803 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/121e68c7-16d6-4c5a-a36b-9149e3561efc-trusted-ca\") pod \"121e68c7-16d6-4c5a-a36b-9149e3561efc\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " Jul 06 19:50:44.230981 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.230843 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/121e68c7-16d6-4c5a-a36b-9149e3561efc-installation-pull-secrets\") pod \"121e68c7-16d6-4c5a-a36b-9149e3561efc\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " Jul 06 19:50:44.230981 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.230859 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-bound-sa-token\") pod \"121e68c7-16d6-4c5a-a36b-9149e3561efc\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " Jul 06 19:50:44.230981 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.230900 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/121e68c7-16d6-4c5a-a36b-9149e3561efc-image-registry-private-configuration\") pod \"121e68c7-16d6-4c5a-a36b-9149e3561efc\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " Jul 06 19:50:44.230981 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.230925 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/121e68c7-16d6-4c5a-a36b-9149e3561efc-ca-trust-extracted\") pod \"121e68c7-16d6-4c5a-a36b-9149e3561efc\" (UID: \"121e68c7-16d6-4c5a-a36b-9149e3561efc\") " Jul 06 19:50:44.231275 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.231183 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "121e68c7-16d6-4c5a-a36b-9149e3561efc" (UID: "121e68c7-16d6-4c5a-a36b-9149e3561efc"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:50:44.231431 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.231398 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/121e68c7-16d6-4c5a-a36b-9149e3561efc-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "121e68c7-16d6-4c5a-a36b-9149e3561efc" (UID: "121e68c7-16d6-4c5a-a36b-9149e3561efc"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:50:44.233256 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.233222 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "121e68c7-16d6-4c5a-a36b-9149e3561efc" (UID: "121e68c7-16d6-4c5a-a36b-9149e3561efc"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:50:44.233256 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.233236 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/121e68c7-16d6-4c5a-a36b-9149e3561efc-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "121e68c7-16d6-4c5a-a36b-9149e3561efc" (UID: "121e68c7-16d6-4c5a-a36b-9149e3561efc"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:50:44.233614 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.233589 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "121e68c7-16d6-4c5a-a36b-9149e3561efc" (UID: "121e68c7-16d6-4c5a-a36b-9149e3561efc"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:50:44.233681 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.233632 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/121e68c7-16d6-4c5a-a36b-9149e3561efc-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "121e68c7-16d6-4c5a-a36b-9149e3561efc" (UID: "121e68c7-16d6-4c5a-a36b-9149e3561efc"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:50:44.233721 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.233687 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-kube-api-access-zjpjg" (OuterVolumeSpecName: "kube-api-access-zjpjg") pod "121e68c7-16d6-4c5a-a36b-9149e3561efc" (UID: "121e68c7-16d6-4c5a-a36b-9149e3561efc"). InnerVolumeSpecName "kube-api-access-zjpjg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:50:44.239905 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.239880 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/121e68c7-16d6-4c5a-a36b-9149e3561efc-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "121e68c7-16d6-4c5a-a36b-9149e3561efc" (UID: "121e68c7-16d6-4c5a-a36b-9149e3561efc"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:50:44.332132 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.332096 2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/121e68c7-16d6-4c5a-a36b-9149e3561efc-ca-trust-extracted\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:50:44.332132 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.332124 2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-certificates\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:50:44.332132 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.332134 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjpjg\" (UniqueName: \"kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-kube-api-access-zjpjg\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:50:44.332391 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.332143 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-registry-tls\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:50:44.332391 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.332154 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/121e68c7-16d6-4c5a-a36b-9149e3561efc-trusted-ca\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:50:44.332391 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.332162 2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/121e68c7-16d6-4c5a-a36b-9149e3561efc-installation-pull-secrets\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:50:44.332391 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.332193 2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/121e68c7-16d6-4c5a-a36b-9149e3561efc-bound-sa-token\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:50:44.332391 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.332202 2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/121e68c7-16d6-4c5a-a36b-9149e3561efc-image-registry-private-configuration\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:50:44.758971 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.758938 2581 generic.go:356] "Generic (PLEG): container finished" podID="121e68c7-16d6-4c5a-a36b-9149e3561efc" containerID="d2048ff618a49c90da922ef8988e952136af12fc0dec07aedfb21c66cbe2eca4" exitCode=0 Jul 06 19:50:44.759126 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.758977 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" Jul 06 19:50:44.759126 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.758982 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" event={"ID":"121e68c7-16d6-4c5a-a36b-9149e3561efc","Type":"ContainerDied","Data":"d2048ff618a49c90da922ef8988e952136af12fc0dec07aedfb21c66cbe2eca4"} Jul 06 19:50:44.759126 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.759023 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65ddd84d6b-j244t" event={"ID":"121e68c7-16d6-4c5a-a36b-9149e3561efc","Type":"ContainerDied","Data":"27a08bd3a28a93e9d07ae1f25fc3e7222cd76ea52eb5765e22cda8c1013afd07"} Jul 06 19:50:44.759126 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.759042 2581 scope.go:117] "RemoveContainer" containerID="d2048ff618a49c90da922ef8988e952136af12fc0dec07aedfb21c66cbe2eca4" Jul 06 19:50:44.767289 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.767276 2581 scope.go:117] "RemoveContainer" containerID="d2048ff618a49c90da922ef8988e952136af12fc0dec07aedfb21c66cbe2eca4" Jul 06 19:50:44.767550 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:50:44.767530 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2048ff618a49c90da922ef8988e952136af12fc0dec07aedfb21c66cbe2eca4\": container with ID starting with d2048ff618a49c90da922ef8988e952136af12fc0dec07aedfb21c66cbe2eca4 not found: ID does not exist" containerID="d2048ff618a49c90da922ef8988e952136af12fc0dec07aedfb21c66cbe2eca4" Jul 06 19:50:44.767597 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.767556 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2048ff618a49c90da922ef8988e952136af12fc0dec07aedfb21c66cbe2eca4"} err="failed to get container status \"d2048ff618a49c90da922ef8988e952136af12fc0dec07aedfb21c66cbe2eca4\": rpc error: code = NotFound desc = could not find container \"d2048ff618a49c90da922ef8988e952136af12fc0dec07aedfb21c66cbe2eca4\": container with ID starting with d2048ff618a49c90da922ef8988e952136af12fc0dec07aedfb21c66cbe2eca4 not found: ID does not exist" Jul 06 19:50:44.781635 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.781612 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65ddd84d6b-j244t"] Jul 06 19:50:44.787073 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:44.787047 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-65ddd84d6b-j244t"] Jul 06 19:50:45.094128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:50:45.094051 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="121e68c7-16d6-4c5a-a36b-9149e3561efc" path="/var/lib/kubelet/pods/121e68c7-16d6-4c5a-a36b-9149e3561efc/volumes" Jul 06 19:51:16.519535 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:16.519436 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:16.538179 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:16.538153 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:16.904174 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:16.904096 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:30.282511 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:30.282472 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:51:30.283022 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:30.282770 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="alertmanager" containerID="cri-o://ca3626efd58d37c1801379a7d596ce7ccbc8ff2a1d4b1896673a70f4aaa59282" gracePeriod=120 Jul 06 19:51:30.283022 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:30.282842 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="kube-rbac-proxy-metric" containerID="cri-o://420b942943c397d1cd081282c500931770ec375f2d84d0d739228c10e8a3e2b4" gracePeriod=120 Jul 06 19:51:30.283022 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:30.282897 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="kube-rbac-proxy-web" containerID="cri-o://f62ac8465eb0bb1dc546d17603abf4b704c74f6633796d54ba54d124dec31dd5" gracePeriod=120 Jul 06 19:51:30.283022 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:30.282926 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="config-reloader" containerID="cri-o://c413d5e8e78ef3e53bf6c2a1553bec611d407f46d98b3de551979a05f572dc78" gracePeriod=120 Jul 06 19:51:30.283022 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:30.282934 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="prom-label-proxy" containerID="cri-o://4d17dad36ffd917f9de302d9a294af52e099bbd91c7d7055f95aae841bee7c78" gracePeriod=120 Jul 06 19:51:30.283022 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:30.282931 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="kube-rbac-proxy" containerID="cri-o://8f1936f4195b2847ab0ef52d23f72c33a7af355954aeb457a768e2c1225b0228" gracePeriod=120 Jul 06 19:51:30.944515 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:30.944486 2581 generic.go:356] "Generic (PLEG): container finished" podID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerID="4d17dad36ffd917f9de302d9a294af52e099bbd91c7d7055f95aae841bee7c78" exitCode=0 Jul 06 19:51:30.944515 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:30.944508 2581 generic.go:356] "Generic (PLEG): container finished" podID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerID="8f1936f4195b2847ab0ef52d23f72c33a7af355954aeb457a768e2c1225b0228" exitCode=0 Jul 06 19:51:30.944515 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:30.944514 2581 generic.go:356] "Generic (PLEG): container finished" podID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerID="c413d5e8e78ef3e53bf6c2a1553bec611d407f46d98b3de551979a05f572dc78" exitCode=0 Jul 06 19:51:30.944515 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:30.944520 2581 generic.go:356] "Generic (PLEG): container finished" podID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerID="ca3626efd58d37c1801379a7d596ce7ccbc8ff2a1d4b1896673a70f4aaa59282" exitCode=0 Jul 06 19:51:30.944777 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:30.944547 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0eaec17-213b-4d1e-8aa8-7fe5013997e0","Type":"ContainerDied","Data":"4d17dad36ffd917f9de302d9a294af52e099bbd91c7d7055f95aae841bee7c78"} Jul 06 19:51:30.944777 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:30.944599 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0eaec17-213b-4d1e-8aa8-7fe5013997e0","Type":"ContainerDied","Data":"8f1936f4195b2847ab0ef52d23f72c33a7af355954aeb457a768e2c1225b0228"} Jul 06 19:51:30.944777 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:30.944624 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0eaec17-213b-4d1e-8aa8-7fe5013997e0","Type":"ContainerDied","Data":"c413d5e8e78ef3e53bf6c2a1553bec611d407f46d98b3de551979a05f572dc78"} Jul 06 19:51:30.944777 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:30.944647 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0eaec17-213b-4d1e-8aa8-7fe5013997e0","Type":"ContainerDied","Data":"ca3626efd58d37c1801379a7d596ce7ccbc8ff2a1d4b1896673a70f4aaa59282"} Jul 06 19:51:31.537739 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.537716 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:31.641455 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.641364 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-cluster-tls-config\") pod \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " Jul 06 19:51:31.641455 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.641404 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-metrics-client-ca\") pod \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " Jul 06 19:51:31.641455 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.641444 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-main-tls\") pod \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " Jul 06 19:51:31.641716 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.641473 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " Jul 06 19:51:31.641716 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.641515 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-config-out\") pod \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " Jul 06 19:51:31.641716 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.641664 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-web-config\") pod \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " Jul 06 19:51:31.641716 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.641709 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-alertmanager-main-db\") pod \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " Jul 06 19:51:31.641936 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.641740 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-alertmanager-trusted-ca-bundle\") pod \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " Jul 06 19:51:31.641936 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.641780 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-tls-assets\") pod \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " Jul 06 19:51:31.641936 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.641825 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9bd2\" (UniqueName: \"kubernetes.io/projected/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-kube-api-access-z9bd2\") pod \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " Jul 06 19:51:31.641936 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.641834 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "c0eaec17-213b-4d1e-8aa8-7fe5013997e0" (UID: "c0eaec17-213b-4d1e-8aa8-7fe5013997e0"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:51:31.641936 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.641912 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-kube-rbac-proxy\") pod \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " Jul 06 19:51:31.642182 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.641944 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-config-volume\") pod \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " Jul 06 19:51:31.642182 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.641991 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-kube-rbac-proxy-web\") pod \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\" (UID: \"c0eaec17-213b-4d1e-8aa8-7fe5013997e0\") " Jul 06 19:51:31.642292 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.642281 2581 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-metrics-client-ca\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:31.643024 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.642995 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "c0eaec17-213b-4d1e-8aa8-7fe5013997e0" (UID: "c0eaec17-213b-4d1e-8aa8-7fe5013997e0"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:51:31.643236 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.643209 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "c0eaec17-213b-4d1e-8aa8-7fe5013997e0" (UID: "c0eaec17-213b-4d1e-8aa8-7fe5013997e0"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:51:31.644391 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.644354 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "c0eaec17-213b-4d1e-8aa8-7fe5013997e0" (UID: "c0eaec17-213b-4d1e-8aa8-7fe5013997e0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:51:31.644636 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.644602 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-config-out" (OuterVolumeSpecName: "config-out") pod "c0eaec17-213b-4d1e-8aa8-7fe5013997e0" (UID: "c0eaec17-213b-4d1e-8aa8-7fe5013997e0"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:51:31.644757 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.644734 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "c0eaec17-213b-4d1e-8aa8-7fe5013997e0" (UID: "c0eaec17-213b-4d1e-8aa8-7fe5013997e0"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:51:31.644831 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.644781 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "c0eaec17-213b-4d1e-8aa8-7fe5013997e0" (UID: "c0eaec17-213b-4d1e-8aa8-7fe5013997e0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:51:31.645210 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.645183 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-kube-api-access-z9bd2" (OuterVolumeSpecName: "kube-api-access-z9bd2") pod "c0eaec17-213b-4d1e-8aa8-7fe5013997e0" (UID: "c0eaec17-213b-4d1e-8aa8-7fe5013997e0"). InnerVolumeSpecName "kube-api-access-z9bd2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:51:31.645312 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.645244 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "c0eaec17-213b-4d1e-8aa8-7fe5013997e0" (UID: "c0eaec17-213b-4d1e-8aa8-7fe5013997e0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:51:31.646286 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.646256 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c0eaec17-213b-4d1e-8aa8-7fe5013997e0" (UID: "c0eaec17-213b-4d1e-8aa8-7fe5013997e0"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:51:31.646445 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.646429 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-config-volume" (OuterVolumeSpecName: "config-volume") pod "c0eaec17-213b-4d1e-8aa8-7fe5013997e0" (UID: "c0eaec17-213b-4d1e-8aa8-7fe5013997e0"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:51:31.648675 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.648654 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "c0eaec17-213b-4d1e-8aa8-7fe5013997e0" (UID: "c0eaec17-213b-4d1e-8aa8-7fe5013997e0"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:51:31.655021 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.655001 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-web-config" (OuterVolumeSpecName: "web-config") pod "c0eaec17-213b-4d1e-8aa8-7fe5013997e0" (UID: "c0eaec17-213b-4d1e-8aa8-7fe5013997e0"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:51:31.742860 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.742837 2581 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-config-volume\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:31.742860 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.742861 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:31.742997 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.742872 2581 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-cluster-tls-config\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:31.742997 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.742883 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-main-tls\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:31.742997 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.742892 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:31.742997 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.742902 2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-config-out\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:31.742997 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.742910 2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-web-config\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:31.742997 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.742918 2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-alertmanager-main-db\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:31.742997 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.742927 2581 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:31.742997 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.742934 2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-tls-assets\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:31.742997 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.742942 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9bd2\" (UniqueName: \"kubernetes.io/projected/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-kube-api-access-z9bd2\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:31.742997 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.742950 2581 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c0eaec17-213b-4d1e-8aa8-7fe5013997e0-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:31.951046 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.950970 2581 generic.go:356] "Generic (PLEG): container finished" podID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerID="420b942943c397d1cd081282c500931770ec375f2d84d0d739228c10e8a3e2b4" exitCode=0 Jul 06 19:51:31.951046 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.950996 2581 generic.go:356] "Generic (PLEG): container finished" podID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerID="f62ac8465eb0bb1dc546d17603abf4b704c74f6633796d54ba54d124dec31dd5" exitCode=0 Jul 06 19:51:31.951046 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.951019 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0eaec17-213b-4d1e-8aa8-7fe5013997e0","Type":"ContainerDied","Data":"420b942943c397d1cd081282c500931770ec375f2d84d0d739228c10e8a3e2b4"} Jul 06 19:51:31.951232 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.951047 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:31.951232 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.951081 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0eaec17-213b-4d1e-8aa8-7fe5013997e0","Type":"ContainerDied","Data":"f62ac8465eb0bb1dc546d17603abf4b704c74f6633796d54ba54d124dec31dd5"} Jul 06 19:51:31.951232 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.951112 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c0eaec17-213b-4d1e-8aa8-7fe5013997e0","Type":"ContainerDied","Data":"584cb714ac21743f5a761850f1911f06221127aacfbfefba9f97ddaea01a00b0"} Jul 06 19:51:31.951232 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.951145 2581 scope.go:117] "RemoveContainer" containerID="4d17dad36ffd917f9de302d9a294af52e099bbd91c7d7055f95aae841bee7c78" Jul 06 19:51:31.958754 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.958735 2581 scope.go:117] "RemoveContainer" containerID="420b942943c397d1cd081282c500931770ec375f2d84d0d739228c10e8a3e2b4" Jul 06 19:51:31.966098 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.966083 2581 scope.go:117] "RemoveContainer" containerID="8f1936f4195b2847ab0ef52d23f72c33a7af355954aeb457a768e2c1225b0228" Jul 06 19:51:31.972846 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.972829 2581 scope.go:117] "RemoveContainer" containerID="f62ac8465eb0bb1dc546d17603abf4b704c74f6633796d54ba54d124dec31dd5" Jul 06 19:51:31.976154 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.976131 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:51:31.981215 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.981198 2581 scope.go:117] "RemoveContainer" containerID="c413d5e8e78ef3e53bf6c2a1553bec611d407f46d98b3de551979a05f572dc78" Jul 06 19:51:31.982343 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.982306 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:51:31.989977 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.989957 2581 scope.go:117] "RemoveContainer" containerID="ca3626efd58d37c1801379a7d596ce7ccbc8ff2a1d4b1896673a70f4aaa59282" Jul 06 19:51:31.996739 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:31.996723 2581 scope.go:117] "RemoveContainer" containerID="e07a84f8b904091680781683bc33f8f6d11849f9e2c6634f563704392347b03c" Jul 06 19:51:32.003935 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.003918 2581 scope.go:117] "RemoveContainer" containerID="4d17dad36ffd917f9de302d9a294af52e099bbd91c7d7055f95aae841bee7c78" Jul 06 19:51:32.004179 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:51:32.004160 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d17dad36ffd917f9de302d9a294af52e099bbd91c7d7055f95aae841bee7c78\": container with ID starting with 4d17dad36ffd917f9de302d9a294af52e099bbd91c7d7055f95aae841bee7c78 not found: ID does not exist" containerID="4d17dad36ffd917f9de302d9a294af52e099bbd91c7d7055f95aae841bee7c78" Jul 06 19:51:32.004225 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.004187 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d17dad36ffd917f9de302d9a294af52e099bbd91c7d7055f95aae841bee7c78"} err="failed to get container status \"4d17dad36ffd917f9de302d9a294af52e099bbd91c7d7055f95aae841bee7c78\": rpc error: code = NotFound desc = could not find container \"4d17dad36ffd917f9de302d9a294af52e099bbd91c7d7055f95aae841bee7c78\": container with ID starting with 4d17dad36ffd917f9de302d9a294af52e099bbd91c7d7055f95aae841bee7c78 not found: ID does not exist" Jul 06 19:51:32.004225 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.004207 2581 scope.go:117] "RemoveContainer" containerID="420b942943c397d1cd081282c500931770ec375f2d84d0d739228c10e8a3e2b4" Jul 06 19:51:32.004493 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:51:32.004474 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"420b942943c397d1cd081282c500931770ec375f2d84d0d739228c10e8a3e2b4\": container with ID starting with 420b942943c397d1cd081282c500931770ec375f2d84d0d739228c10e8a3e2b4 not found: ID does not exist" containerID="420b942943c397d1cd081282c500931770ec375f2d84d0d739228c10e8a3e2b4" Jul 06 19:51:32.004570 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.004504 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"420b942943c397d1cd081282c500931770ec375f2d84d0d739228c10e8a3e2b4"} err="failed to get container status \"420b942943c397d1cd081282c500931770ec375f2d84d0d739228c10e8a3e2b4\": rpc error: code = NotFound desc = could not find container \"420b942943c397d1cd081282c500931770ec375f2d84d0d739228c10e8a3e2b4\": container with ID starting with 420b942943c397d1cd081282c500931770ec375f2d84d0d739228c10e8a3e2b4 not found: ID does not exist" Jul 06 19:51:32.004570 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.004525 2581 scope.go:117] "RemoveContainer" containerID="8f1936f4195b2847ab0ef52d23f72c33a7af355954aeb457a768e2c1225b0228" Jul 06 19:51:32.004795 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:51:32.004771 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f1936f4195b2847ab0ef52d23f72c33a7af355954aeb457a768e2c1225b0228\": container with ID starting with 8f1936f4195b2847ab0ef52d23f72c33a7af355954aeb457a768e2c1225b0228 not found: ID does not exist" containerID="8f1936f4195b2847ab0ef52d23f72c33a7af355954aeb457a768e2c1225b0228" Jul 06 19:51:32.004832 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.004800 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f1936f4195b2847ab0ef52d23f72c33a7af355954aeb457a768e2c1225b0228"} err="failed to get container status \"8f1936f4195b2847ab0ef52d23f72c33a7af355954aeb457a768e2c1225b0228\": rpc error: code = NotFound desc = could not find container \"8f1936f4195b2847ab0ef52d23f72c33a7af355954aeb457a768e2c1225b0228\": container with ID starting with 8f1936f4195b2847ab0ef52d23f72c33a7af355954aeb457a768e2c1225b0228 not found: ID does not exist" Jul 06 19:51:32.004832 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.004821 2581 scope.go:117] "RemoveContainer" containerID="f62ac8465eb0bb1dc546d17603abf4b704c74f6633796d54ba54d124dec31dd5" Jul 06 19:51:32.005030 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:51:32.005016 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f62ac8465eb0bb1dc546d17603abf4b704c74f6633796d54ba54d124dec31dd5\": container with ID starting with f62ac8465eb0bb1dc546d17603abf4b704c74f6633796d54ba54d124dec31dd5 not found: ID does not exist" containerID="f62ac8465eb0bb1dc546d17603abf4b704c74f6633796d54ba54d124dec31dd5" Jul 06 19:51:32.005069 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.005033 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f62ac8465eb0bb1dc546d17603abf4b704c74f6633796d54ba54d124dec31dd5"} err="failed to get container status \"f62ac8465eb0bb1dc546d17603abf4b704c74f6633796d54ba54d124dec31dd5\": rpc error: code = NotFound desc = could not find container \"f62ac8465eb0bb1dc546d17603abf4b704c74f6633796d54ba54d124dec31dd5\": container with ID starting with f62ac8465eb0bb1dc546d17603abf4b704c74f6633796d54ba54d124dec31dd5 not found: ID does not exist" Jul 06 19:51:32.005069 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.005045 2581 scope.go:117] "RemoveContainer" containerID="c413d5e8e78ef3e53bf6c2a1553bec611d407f46d98b3de551979a05f572dc78" Jul 06 19:51:32.005241 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:51:32.005224 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c413d5e8e78ef3e53bf6c2a1553bec611d407f46d98b3de551979a05f572dc78\": container with ID starting with c413d5e8e78ef3e53bf6c2a1553bec611d407f46d98b3de551979a05f572dc78 not found: ID does not exist" containerID="c413d5e8e78ef3e53bf6c2a1553bec611d407f46d98b3de551979a05f572dc78" Jul 06 19:51:32.005281 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.005243 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c413d5e8e78ef3e53bf6c2a1553bec611d407f46d98b3de551979a05f572dc78"} err="failed to get container status \"c413d5e8e78ef3e53bf6c2a1553bec611d407f46d98b3de551979a05f572dc78\": rpc error: code = NotFound desc = could not find container \"c413d5e8e78ef3e53bf6c2a1553bec611d407f46d98b3de551979a05f572dc78\": container with ID starting with c413d5e8e78ef3e53bf6c2a1553bec611d407f46d98b3de551979a05f572dc78 not found: ID does not exist" Jul 06 19:51:32.005281 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.005255 2581 scope.go:117] "RemoveContainer" containerID="ca3626efd58d37c1801379a7d596ce7ccbc8ff2a1d4b1896673a70f4aaa59282" Jul 06 19:51:32.005487 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:51:32.005472 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca3626efd58d37c1801379a7d596ce7ccbc8ff2a1d4b1896673a70f4aaa59282\": container with ID starting with ca3626efd58d37c1801379a7d596ce7ccbc8ff2a1d4b1896673a70f4aaa59282 not found: ID does not exist" containerID="ca3626efd58d37c1801379a7d596ce7ccbc8ff2a1d4b1896673a70f4aaa59282" Jul 06 19:51:32.005535 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.005492 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca3626efd58d37c1801379a7d596ce7ccbc8ff2a1d4b1896673a70f4aaa59282"} err="failed to get container status \"ca3626efd58d37c1801379a7d596ce7ccbc8ff2a1d4b1896673a70f4aaa59282\": rpc error: code = NotFound desc = could not find container \"ca3626efd58d37c1801379a7d596ce7ccbc8ff2a1d4b1896673a70f4aaa59282\": container with ID starting with ca3626efd58d37c1801379a7d596ce7ccbc8ff2a1d4b1896673a70f4aaa59282 not found: ID does not exist" Jul 06 19:51:32.005535 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.005504 2581 scope.go:117] "RemoveContainer" containerID="e07a84f8b904091680781683bc33f8f6d11849f9e2c6634f563704392347b03c" Jul 06 19:51:32.005709 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:51:32.005691 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e07a84f8b904091680781683bc33f8f6d11849f9e2c6634f563704392347b03c\": container with ID starting with e07a84f8b904091680781683bc33f8f6d11849f9e2c6634f563704392347b03c not found: ID does not exist" containerID="e07a84f8b904091680781683bc33f8f6d11849f9e2c6634f563704392347b03c" Jul 06 19:51:32.005754 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.005712 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e07a84f8b904091680781683bc33f8f6d11849f9e2c6634f563704392347b03c"} err="failed to get container status \"e07a84f8b904091680781683bc33f8f6d11849f9e2c6634f563704392347b03c\": rpc error: code = NotFound desc = could not find container \"e07a84f8b904091680781683bc33f8f6d11849f9e2c6634f563704392347b03c\": container with ID starting with e07a84f8b904091680781683bc33f8f6d11849f9e2c6634f563704392347b03c not found: ID does not exist" Jul 06 19:51:32.005754 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.005730 2581 scope.go:117] "RemoveContainer" containerID="4d17dad36ffd917f9de302d9a294af52e099bbd91c7d7055f95aae841bee7c78" Jul 06 19:51:32.005916 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.005895 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d17dad36ffd917f9de302d9a294af52e099bbd91c7d7055f95aae841bee7c78"} err="failed to get container status \"4d17dad36ffd917f9de302d9a294af52e099bbd91c7d7055f95aae841bee7c78\": rpc error: code = NotFound desc = could not find container \"4d17dad36ffd917f9de302d9a294af52e099bbd91c7d7055f95aae841bee7c78\": container with ID starting with 4d17dad36ffd917f9de302d9a294af52e099bbd91c7d7055f95aae841bee7c78 not found: ID does not exist" Jul 06 19:51:32.005916 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.005915 2581 scope.go:117] "RemoveContainer" containerID="420b942943c397d1cd081282c500931770ec375f2d84d0d739228c10e8a3e2b4" Jul 06 19:51:32.006087 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.006072 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"420b942943c397d1cd081282c500931770ec375f2d84d0d739228c10e8a3e2b4"} err="failed to get container status \"420b942943c397d1cd081282c500931770ec375f2d84d0d739228c10e8a3e2b4\": rpc error: code = NotFound desc = could not find container \"420b942943c397d1cd081282c500931770ec375f2d84d0d739228c10e8a3e2b4\": container with ID starting with 420b942943c397d1cd081282c500931770ec375f2d84d0d739228c10e8a3e2b4 not found: ID does not exist" Jul 06 19:51:32.006087 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.006087 2581 scope.go:117] "RemoveContainer" containerID="8f1936f4195b2847ab0ef52d23f72c33a7af355954aeb457a768e2c1225b0228" Jul 06 19:51:32.006291 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.006277 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f1936f4195b2847ab0ef52d23f72c33a7af355954aeb457a768e2c1225b0228"} err="failed to get container status \"8f1936f4195b2847ab0ef52d23f72c33a7af355954aeb457a768e2c1225b0228\": rpc error: code = NotFound desc = could not find container \"8f1936f4195b2847ab0ef52d23f72c33a7af355954aeb457a768e2c1225b0228\": container with ID starting with 8f1936f4195b2847ab0ef52d23f72c33a7af355954aeb457a768e2c1225b0228 not found: ID does not exist" Jul 06 19:51:32.006291 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.006290 2581 scope.go:117] "RemoveContainer" containerID="f62ac8465eb0bb1dc546d17603abf4b704c74f6633796d54ba54d124dec31dd5" Jul 06 19:51:32.006525 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.006507 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f62ac8465eb0bb1dc546d17603abf4b704c74f6633796d54ba54d124dec31dd5"} err="failed to get container status \"f62ac8465eb0bb1dc546d17603abf4b704c74f6633796d54ba54d124dec31dd5\": rpc error: code = NotFound desc = could not find container \"f62ac8465eb0bb1dc546d17603abf4b704c74f6633796d54ba54d124dec31dd5\": container with ID starting with f62ac8465eb0bb1dc546d17603abf4b704c74f6633796d54ba54d124dec31dd5 not found: ID does not exist" Jul 06 19:51:32.006588 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.006527 2581 scope.go:117] "RemoveContainer" containerID="c413d5e8e78ef3e53bf6c2a1553bec611d407f46d98b3de551979a05f572dc78" Jul 06 19:51:32.006757 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.006737 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c413d5e8e78ef3e53bf6c2a1553bec611d407f46d98b3de551979a05f572dc78"} err="failed to get container status \"c413d5e8e78ef3e53bf6c2a1553bec611d407f46d98b3de551979a05f572dc78\": rpc error: code = NotFound desc = could not find container \"c413d5e8e78ef3e53bf6c2a1553bec611d407f46d98b3de551979a05f572dc78\": container with ID starting with c413d5e8e78ef3e53bf6c2a1553bec611d407f46d98b3de551979a05f572dc78 not found: ID does not exist" Jul 06 19:51:32.006804 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.006758 2581 scope.go:117] "RemoveContainer" containerID="ca3626efd58d37c1801379a7d596ce7ccbc8ff2a1d4b1896673a70f4aaa59282" Jul 06 19:51:32.006953 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.006936 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca3626efd58d37c1801379a7d596ce7ccbc8ff2a1d4b1896673a70f4aaa59282"} err="failed to get container status \"ca3626efd58d37c1801379a7d596ce7ccbc8ff2a1d4b1896673a70f4aaa59282\": rpc error: code = NotFound desc = could not find container \"ca3626efd58d37c1801379a7d596ce7ccbc8ff2a1d4b1896673a70f4aaa59282\": container with ID starting with ca3626efd58d37c1801379a7d596ce7ccbc8ff2a1d4b1896673a70f4aaa59282 not found: ID does not exist" Jul 06 19:51:32.006989 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.006954 2581 scope.go:117] "RemoveContainer" containerID="e07a84f8b904091680781683bc33f8f6d11849f9e2c6634f563704392347b03c" Jul 06 19:51:32.007173 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.007156 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e07a84f8b904091680781683bc33f8f6d11849f9e2c6634f563704392347b03c"} err="failed to get container status \"e07a84f8b904091680781683bc33f8f6d11849f9e2c6634f563704392347b03c\": rpc error: code = NotFound desc = could not find container \"e07a84f8b904091680781683bc33f8f6d11849f9e2c6634f563704392347b03c\": container with ID starting with e07a84f8b904091680781683bc33f8f6d11849f9e2c6634f563704392347b03c not found: ID does not exist" Jul 06 19:51:32.011061 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011040 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:51:32.011426 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011409 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="alertmanager" Jul 06 19:51:32.011541 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011428 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="alertmanager" Jul 06 19:51:32.011541 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011439 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="kube-rbac-proxy-web" Jul 06 19:51:32.011541 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011447 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="kube-rbac-proxy-web" Jul 06 19:51:32.011541 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011466 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="config-reloader" Jul 06 19:51:32.011541 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011474 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="config-reloader" Jul 06 19:51:32.011541 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011487 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="kube-rbac-proxy" Jul 06 19:51:32.011541 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011496 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="kube-rbac-proxy" Jul 06 19:51:32.011541 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011509 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="121e68c7-16d6-4c5a-a36b-9149e3561efc" containerName="registry" Jul 06 19:51:32.011541 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011517 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="121e68c7-16d6-4c5a-a36b-9149e3561efc" containerName="registry" Jul 06 19:51:32.011541 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011530 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="kube-rbac-proxy-metric" Jul 06 19:51:32.011541 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011538 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="kube-rbac-proxy-metric" Jul 06 19:51:32.012053 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011549 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="init-config-reloader" Jul 06 19:51:32.012053 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011557 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="init-config-reloader" Jul 06 19:51:32.012053 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011574 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="prom-label-proxy" Jul 06 19:51:32.012053 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011583 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="prom-label-proxy" Jul 06 19:51:32.012053 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011684 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="config-reloader" Jul 06 19:51:32.012053 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011698 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="kube-rbac-proxy" Jul 06 19:51:32.012053 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011707 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="prom-label-proxy" Jul 06 19:51:32.012053 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011717 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="kube-rbac-proxy-metric" Jul 06 19:51:32.012053 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011726 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="alertmanager" Jul 06 19:51:32.012053 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011736 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="121e68c7-16d6-4c5a-a36b-9149e3561efc" containerName="registry" Jul 06 19:51:32.012053 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.011745 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" containerName="kube-rbac-proxy-web" Jul 06 19:51:32.019388 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.019369 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.021893 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.021873 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 19:51:32.022106 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.022090 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-vzkkh\"" Jul 06 19:51:32.022237 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.022220 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 19:51:32.022483 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.022469 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 19:51:32.022542 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.022503 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 19:51:32.022663 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.022640 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 19:51:32.022747 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.022687 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 19:51:32.022747 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.022710 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 19:51:32.022856 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.022772 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 19:51:32.027956 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.027935 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 19:51:32.030567 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.030548 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:51:32.146715 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.146680 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1687c4c2-7af9-479e-a31f-dbc13a04c41c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.146715 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.146719 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1687c4c2-7af9-479e-a31f-dbc13a04c41c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.146942 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.146739 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1687c4c2-7af9-479e-a31f-dbc13a04c41c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.146942 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.146756 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1687c4c2-7af9-479e-a31f-dbc13a04c41c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.146942 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.146773 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1687c4c2-7af9-479e-a31f-dbc13a04c41c-config-volume\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.146942 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.146823 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1687c4c2-7af9-479e-a31f-dbc13a04c41c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.146942 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.146856 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1687c4c2-7af9-479e-a31f-dbc13a04c41c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.146942 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.146878 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1687c4c2-7af9-479e-a31f-dbc13a04c41c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.146942 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.146893 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1687c4c2-7af9-479e-a31f-dbc13a04c41c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.146942 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.146908 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkw25\" (UniqueName: \"kubernetes.io/projected/1687c4c2-7af9-479e-a31f-dbc13a04c41c-kube-api-access-jkw25\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.147175 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.146949 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1687c4c2-7af9-479e-a31f-dbc13a04c41c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.147175 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.146968 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1687c4c2-7af9-479e-a31f-dbc13a04c41c-web-config\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.147175 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.147012 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1687c4c2-7af9-479e-a31f-dbc13a04c41c-config-out\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.247693 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.247608 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1687c4c2-7af9-479e-a31f-dbc13a04c41c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.247693 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.247652 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1687c4c2-7af9-479e-a31f-dbc13a04c41c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.247693 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.247671 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkw25\" (UniqueName: \"kubernetes.io/projected/1687c4c2-7af9-479e-a31f-dbc13a04c41c-kube-api-access-jkw25\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.247950 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.247704 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1687c4c2-7af9-479e-a31f-dbc13a04c41c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.247950 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.247724 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1687c4c2-7af9-479e-a31f-dbc13a04c41c-web-config\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.247950 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.247753 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1687c4c2-7af9-479e-a31f-dbc13a04c41c-config-out\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.247950 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.247784 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1687c4c2-7af9-479e-a31f-dbc13a04c41c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.247950 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.247865 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1687c4c2-7af9-479e-a31f-dbc13a04c41c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.247950 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.247893 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1687c4c2-7af9-479e-a31f-dbc13a04c41c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.247950 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.247917 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1687c4c2-7af9-479e-a31f-dbc13a04c41c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.247950 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.247945 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1687c4c2-7af9-479e-a31f-dbc13a04c41c-config-volume\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.248408 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.247973 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1687c4c2-7af9-479e-a31f-dbc13a04c41c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.248408 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.248042 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1687c4c2-7af9-479e-a31f-dbc13a04c41c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.249194 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.249165 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1687c4c2-7af9-479e-a31f-dbc13a04c41c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.250589 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.250561 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1687c4c2-7af9-479e-a31f-dbc13a04c41c-config-out\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.250711 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.250670 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1687c4c2-7af9-479e-a31f-dbc13a04c41c-web-config\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.250711 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.250673 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1687c4c2-7af9-479e-a31f-dbc13a04c41c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.250850 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.250819 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1687c4c2-7af9-479e-a31f-dbc13a04c41c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.251069 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.251037 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1687c4c2-7af9-479e-a31f-dbc13a04c41c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.251150 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.251132 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1687c4c2-7af9-479e-a31f-dbc13a04c41c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.251294 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.251274 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1687c4c2-7af9-479e-a31f-dbc13a04c41c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.251414 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.251390 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1687c4c2-7af9-479e-a31f-dbc13a04c41c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.251501 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.251395 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1687c4c2-7af9-479e-a31f-dbc13a04c41c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.251682 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.251664 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1687c4c2-7af9-479e-a31f-dbc13a04c41c-config-volume\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.252677 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.252661 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1687c4c2-7af9-479e-a31f-dbc13a04c41c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.260381 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.260362 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkw25\" (UniqueName: \"kubernetes.io/projected/1687c4c2-7af9-479e-a31f-dbc13a04c41c-kube-api-access-jkw25\") pod \"alertmanager-main-0\" (UID: \"1687c4c2-7af9-479e-a31f-dbc13a04c41c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.329416 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.329389 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 19:51:32.466211 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.466185 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 19:51:32.468155 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:51:32.468128 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1687c4c2_7af9_479e_a31f_dbc13a04c41c.slice/crio-c12fa732b792b395f16309dcbee330135ddd8dd05e462b1433e4f79e78bc84ae WatchSource:0}: Error finding container c12fa732b792b395f16309dcbee330135ddd8dd05e462b1433e4f79e78bc84ae: Status 404 returned error can't find the container with id c12fa732b792b395f16309dcbee330135ddd8dd05e462b1433e4f79e78bc84ae Jul 06 19:51:32.956625 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.956595 2581 generic.go:356] "Generic (PLEG): container finished" podID="1687c4c2-7af9-479e-a31f-dbc13a04c41c" containerID="75989f1493c72aa2779e7e5c4cf85d3edc741d2296cd51090d2a48fc969ed4ec" exitCode=0 Jul 06 19:51:32.957032 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.956688 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1687c4c2-7af9-479e-a31f-dbc13a04c41c","Type":"ContainerDied","Data":"75989f1493c72aa2779e7e5c4cf85d3edc741d2296cd51090d2a48fc969ed4ec"} Jul 06 19:51:32.957032 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:32.956767 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1687c4c2-7af9-479e-a31f-dbc13a04c41c","Type":"ContainerStarted","Data":"c12fa732b792b395f16309dcbee330135ddd8dd05e462b1433e4f79e78bc84ae"} Jul 06 19:51:33.095374 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:33.095351 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0eaec17-213b-4d1e-8aa8-7fe5013997e0" path="/var/lib/kubelet/pods/c0eaec17-213b-4d1e-8aa8-7fe5013997e0/volumes" Jul 06 19:51:33.969239 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:33.969201 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1687c4c2-7af9-479e-a31f-dbc13a04c41c","Type":"ContainerStarted","Data":"8a1a1de838f2c272adb3d638ff79ca7510673ee2c4a9b7d16e9aaf76074851e1"} Jul 06 19:51:33.969710 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:33.969285 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1687c4c2-7af9-479e-a31f-dbc13a04c41c","Type":"ContainerStarted","Data":"f1dc171a3a3035b4e147740240956444cae9ca8a383a8eae0edc636508f129be"} Jul 06 19:51:33.969710 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:33.969312 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1687c4c2-7af9-479e-a31f-dbc13a04c41c","Type":"ContainerStarted","Data":"c816efeadc1dd968df931a0e6d8f90a8b108a5c16350e753d43fb7c5823e7925"} Jul 06 19:51:33.969710 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:33.969358 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1687c4c2-7af9-479e-a31f-dbc13a04c41c","Type":"ContainerStarted","Data":"208938f40e63c8e312710e3b6aa6aea6a8bbb5009047468c52b823e161da4a15"} Jul 06 19:51:33.969710 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:33.969382 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1687c4c2-7af9-479e-a31f-dbc13a04c41c","Type":"ContainerStarted","Data":"a28fc3f8b412c21aa35233810042d5876c5d9dd9e974cd1104f239ec0bf7d8f1"} Jul 06 19:51:33.969710 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:33.969405 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1687c4c2-7af9-479e-a31f-dbc13a04c41c","Type":"ContainerStarted","Data":"0c3e060f82ed4058b8a756b5990e03098f9e6a1d752de62d66fc73d1cc349f3b"} Jul 06 19:51:33.999338 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:33.999141 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.98408229 podStartE2EDuration="2.999122595s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.040305ms" isStatefulPod=false podCreationTimestamp="2026-07-06 19:51:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:51:33.998608856 +0000 UTC m=+193.445845575" watchObservedRunningTime="2026-07-06 19:51:33.999122595 +0000 UTC m=+193.446359313" Jul 06 19:51:34.532649 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.532610 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:51:34.532992 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.532965 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="kube-rbac-proxy-thanos" containerID="cri-o://254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f" gracePeriod=600 Jul 06 19:51:34.533225 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.532942 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="prometheus" containerID="cri-o://af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e" gracePeriod=600 Jul 06 19:51:34.533314 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.532989 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="config-reloader" containerID="cri-o://467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb" gracePeriod=600 Jul 06 19:51:34.533386 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.532961 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="kube-rbac-proxy" containerID="cri-o://f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76" gracePeriod=600 Jul 06 19:51:34.533441 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.533009 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="kube-rbac-proxy-web" containerID="cri-o://b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088" gracePeriod=600 Jul 06 19:51:34.533441 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.533009 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="thanos-sidecar" containerID="cri-o://bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c" gracePeriod=600 Jul 06 19:51:34.766870 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.766846 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:34.871678 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.871599 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-db\") pod \"458a7e6e-c625-401c-a9b2-75ea36214fe5\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " Jul 06 19:51:34.871678 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.871661 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-configmap-kubelet-serving-ca-bundle\") pod \"458a7e6e-c625-401c-a9b2-75ea36214fe5\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " Jul 06 19:51:34.871911 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.871689 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-config\") pod \"458a7e6e-c625-401c-a9b2-75ea36214fe5\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " Jul 06 19:51:34.871911 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.871732 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-thanos-prometheus-http-client-file\") pod \"458a7e6e-c625-401c-a9b2-75ea36214fe5\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " Jul 06 19:51:34.871911 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.871761 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-rulefiles-1\") pod \"458a7e6e-c625-401c-a9b2-75ea36214fe5\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " Jul 06 19:51:34.871911 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.871788 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-kube-rbac-proxy\") pod \"458a7e6e-c625-401c-a9b2-75ea36214fe5\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " Jul 06 19:51:34.871911 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.871818 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-rulefiles-0\") pod \"458a7e6e-c625-401c-a9b2-75ea36214fe5\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " Jul 06 19:51:34.871911 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.871843 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-web-config\") pod \"458a7e6e-c625-401c-a9b2-75ea36214fe5\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " Jul 06 19:51:34.871911 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.871867 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8zvc\" (UniqueName: \"kubernetes.io/projected/458a7e6e-c625-401c-a9b2-75ea36214fe5-kube-api-access-t8zvc\") pod \"458a7e6e-c625-401c-a9b2-75ea36214fe5\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " Jul 06 19:51:34.871911 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.871908 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-rulefiles-2\") pod \"458a7e6e-c625-401c-a9b2-75ea36214fe5\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " Jul 06 19:51:34.872289 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.871941 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-prometheus-k8s-tls\") pod \"458a7e6e-c625-401c-a9b2-75ea36214fe5\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " Jul 06 19:51:34.872289 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.871975 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-trusted-ca-bundle\") pod \"458a7e6e-c625-401c-a9b2-75ea36214fe5\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " Jul 06 19:51:34.872289 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.872001 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"458a7e6e-c625-401c-a9b2-75ea36214fe5\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " Jul 06 19:51:34.872289 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.872030 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/458a7e6e-c625-401c-a9b2-75ea36214fe5-config-out\") pod \"458a7e6e-c625-401c-a9b2-75ea36214fe5\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " Jul 06 19:51:34.872289 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.872056 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/458a7e6e-c625-401c-a9b2-75ea36214fe5-tls-assets\") pod \"458a7e6e-c625-401c-a9b2-75ea36214fe5\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " Jul 06 19:51:34.872289 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.872084 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"458a7e6e-c625-401c-a9b2-75ea36214fe5\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " Jul 06 19:51:34.872289 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.872121 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-grpc-tls\") pod \"458a7e6e-c625-401c-a9b2-75ea36214fe5\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " Jul 06 19:51:34.872289 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.872125 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "458a7e6e-c625-401c-a9b2-75ea36214fe5" (UID: "458a7e6e-c625-401c-a9b2-75ea36214fe5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:51:34.872289 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.872154 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-configmap-serving-certs-ca-bundle\") pod \"458a7e6e-c625-401c-a9b2-75ea36214fe5\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " Jul 06 19:51:34.872289 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.872155 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "458a7e6e-c625-401c-a9b2-75ea36214fe5" (UID: "458a7e6e-c625-401c-a9b2-75ea36214fe5"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:51:34.872289 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.872201 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-metrics-client-certs\") pod \"458a7e6e-c625-401c-a9b2-75ea36214fe5\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " Jul 06 19:51:34.872289 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.872241 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-configmap-metrics-client-ca\") pod \"458a7e6e-c625-401c-a9b2-75ea36214fe5\" (UID: \"458a7e6e-c625-401c-a9b2-75ea36214fe5\") " Jul 06 19:51:34.872871 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.872539 2581 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:34.872871 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.872556 2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:34.872871 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.872710 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "458a7e6e-c625-401c-a9b2-75ea36214fe5" (UID: "458a7e6e-c625-401c-a9b2-75ea36214fe5"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:51:34.873031 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.873002 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "458a7e6e-c625-401c-a9b2-75ea36214fe5" (UID: "458a7e6e-c625-401c-a9b2-75ea36214fe5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:51:34.873085 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.873045 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "458a7e6e-c625-401c-a9b2-75ea36214fe5" (UID: "458a7e6e-c625-401c-a9b2-75ea36214fe5"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:51:34.873828 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.873712 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "458a7e6e-c625-401c-a9b2-75ea36214fe5" (UID: "458a7e6e-c625-401c-a9b2-75ea36214fe5"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:51:34.873828 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.873792 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "458a7e6e-c625-401c-a9b2-75ea36214fe5" (UID: "458a7e6e-c625-401c-a9b2-75ea36214fe5"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:51:34.874016 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.873931 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "458a7e6e-c625-401c-a9b2-75ea36214fe5" (UID: "458a7e6e-c625-401c-a9b2-75ea36214fe5"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:51:34.875232 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.875204 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "458a7e6e-c625-401c-a9b2-75ea36214fe5" (UID: "458a7e6e-c625-401c-a9b2-75ea36214fe5"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:51:34.875498 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.875457 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "458a7e6e-c625-401c-a9b2-75ea36214fe5" (UID: "458a7e6e-c625-401c-a9b2-75ea36214fe5"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:51:34.875570 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.875536 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-config" (OuterVolumeSpecName: "config") pod "458a7e6e-c625-401c-a9b2-75ea36214fe5" (UID: "458a7e6e-c625-401c-a9b2-75ea36214fe5"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:51:34.875740 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.875709 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/458a7e6e-c625-401c-a9b2-75ea36214fe5-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "458a7e6e-c625-401c-a9b2-75ea36214fe5" (UID: "458a7e6e-c625-401c-a9b2-75ea36214fe5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:51:34.876155 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.876121 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "458a7e6e-c625-401c-a9b2-75ea36214fe5" (UID: "458a7e6e-c625-401c-a9b2-75ea36214fe5"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:51:34.876352 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.876293 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "458a7e6e-c625-401c-a9b2-75ea36214fe5" (UID: "458a7e6e-c625-401c-a9b2-75ea36214fe5"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:51:34.876488 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.876461 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/458a7e6e-c625-401c-a9b2-75ea36214fe5-config-out" (OuterVolumeSpecName: "config-out") pod "458a7e6e-c625-401c-a9b2-75ea36214fe5" (UID: "458a7e6e-c625-401c-a9b2-75ea36214fe5"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:51:34.876783 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.876766 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "458a7e6e-c625-401c-a9b2-75ea36214fe5" (UID: "458a7e6e-c625-401c-a9b2-75ea36214fe5"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:51:34.876821 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.876780 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/458a7e6e-c625-401c-a9b2-75ea36214fe5-kube-api-access-t8zvc" (OuterVolumeSpecName: "kube-api-access-t8zvc") pod "458a7e6e-c625-401c-a9b2-75ea36214fe5" (UID: "458a7e6e-c625-401c-a9b2-75ea36214fe5"). InnerVolumeSpecName "kube-api-access-t8zvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:51:34.876958 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.876945 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "458a7e6e-c625-401c-a9b2-75ea36214fe5" (UID: "458a7e6e-c625-401c-a9b2-75ea36214fe5"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:51:34.877659 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.877636 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "458a7e6e-c625-401c-a9b2-75ea36214fe5" (UID: "458a7e6e-c625-401c-a9b2-75ea36214fe5"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:51:34.886115 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.886097 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-web-config" (OuterVolumeSpecName: "web-config") pod "458a7e6e-c625-401c-a9b2-75ea36214fe5" (UID: "458a7e6e-c625-401c-a9b2-75ea36214fe5"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:51:34.972915 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.972882 2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:34.972915 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.972908 2581 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-web-config\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:34.972915 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.972918 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8zvc\" (UniqueName: \"kubernetes.io/projected/458a7e6e-c625-401c-a9b2-75ea36214fe5-kube-api-access-t8zvc\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:34.973363 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.972926 2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:34.973363 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.972935 2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:34.973363 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.972943 2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:34.973363 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.972957 2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:34.973363 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.972970 2581 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/458a7e6e-c625-401c-a9b2-75ea36214fe5-config-out\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:34.973363 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.972980 2581 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/458a7e6e-c625-401c-a9b2-75ea36214fe5-tls-assets\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:34.973363 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.972987 2581 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:34.973363 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.972994 2581 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-grpc-tls\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:34.973363 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.973003 2581 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:34.973363 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.973012 2581 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-metrics-client-certs\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:34.973363 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.973020 2581 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/458a7e6e-c625-401c-a9b2-75ea36214fe5-configmap-metrics-client-ca\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:34.973363 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.973028 2581 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/458a7e6e-c625-401c-a9b2-75ea36214fe5-prometheus-k8s-db\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:34.973363 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.973037 2581 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-config\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:34.973363 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.973051 2581 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:34.973363 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.973062 2581 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/458a7e6e-c625-401c-a9b2-75ea36214fe5-secret-kube-rbac-proxy\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:51:34.976664 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.976637 2581 generic.go:356] "Generic (PLEG): container finished" podID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerID="254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f" exitCode=0 Jul 06 19:51:34.976729 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.976668 2581 generic.go:356] "Generic (PLEG): container finished" podID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerID="f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76" exitCode=0 Jul 06 19:51:34.976729 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.976679 2581 generic.go:356] "Generic (PLEG): container finished" podID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerID="b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088" exitCode=0 Jul 06 19:51:34.976729 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.976689 2581 generic.go:356] "Generic (PLEG): container finished" podID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerID="bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c" exitCode=0 Jul 06 19:51:34.976729 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.976698 2581 generic.go:356] "Generic (PLEG): container finished" podID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerID="467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb" exitCode=0 Jul 06 19:51:34.976729 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.976706 2581 generic.go:356] "Generic (PLEG): container finished" podID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerID="af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e" exitCode=0 Jul 06 19:51:34.976867 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.976720 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"458a7e6e-c625-401c-a9b2-75ea36214fe5","Type":"ContainerDied","Data":"254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f"} Jul 06 19:51:34.976867 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.976787 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"458a7e6e-c625-401c-a9b2-75ea36214fe5","Type":"ContainerDied","Data":"f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76"} Jul 06 19:51:34.976867 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.976732 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:34.976867 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.976815 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"458a7e6e-c625-401c-a9b2-75ea36214fe5","Type":"ContainerDied","Data":"b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088"} Jul 06 19:51:34.976867 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.976826 2581 scope.go:117] "RemoveContainer" containerID="254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f" Jul 06 19:51:34.976867 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.976841 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"458a7e6e-c625-401c-a9b2-75ea36214fe5","Type":"ContainerDied","Data":"bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c"} Jul 06 19:51:34.976867 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.976867 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"458a7e6e-c625-401c-a9b2-75ea36214fe5","Type":"ContainerDied","Data":"467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb"} Jul 06 19:51:34.977080 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.976891 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"458a7e6e-c625-401c-a9b2-75ea36214fe5","Type":"ContainerDied","Data":"af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e"} Jul 06 19:51:34.977080 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.976917 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"458a7e6e-c625-401c-a9b2-75ea36214fe5","Type":"ContainerDied","Data":"36cccac5d8ba0acc0a94f03e9b20a679744e7d16211e08f6de243e70f3bf4936"} Jul 06 19:51:34.984529 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.984507 2581 scope.go:117] "RemoveContainer" containerID="f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76" Jul 06 19:51:34.991375 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.991308 2581 scope.go:117] "RemoveContainer" containerID="b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088" Jul 06 19:51:34.997861 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:34.997837 2581 scope.go:117] "RemoveContainer" containerID="bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c" Jul 06 19:51:35.000278 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.000260 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:51:35.005508 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.005438 2581 scope.go:117] "RemoveContainer" containerID="467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb" Jul 06 19:51:35.006837 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.006820 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:51:35.011824 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.011808 2581 scope.go:117] "RemoveContainer" containerID="af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e" Jul 06 19:51:35.018770 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.018753 2581 scope.go:117] "RemoveContainer" containerID="1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f" Jul 06 19:51:35.025125 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.025109 2581 scope.go:117] "RemoveContainer" containerID="254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f" Jul 06 19:51:35.025399 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:51:35.025371 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f\": container with ID starting with 254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f not found: ID does not exist" containerID="254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f" Jul 06 19:51:35.025477 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.025398 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f"} err="failed to get container status \"254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f\": rpc error: code = NotFound desc = could not find container \"254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f\": container with ID starting with 254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f not found: ID does not exist" Jul 06 19:51:35.025477 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.025415 2581 scope.go:117] "RemoveContainer" containerID="f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76" Jul 06 19:51:35.025611 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:51:35.025596 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76\": container with ID starting with f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76 not found: ID does not exist" containerID="f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76" Jul 06 19:51:35.025646 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.025615 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76"} err="failed to get container status \"f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76\": rpc error: code = NotFound desc = could not find container \"f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76\": container with ID starting with f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76 not found: ID does not exist" Jul 06 19:51:35.025646 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.025627 2581 scope.go:117] "RemoveContainer" containerID="b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088" Jul 06 19:51:35.025831 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:51:35.025815 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088\": container with ID starting with b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088 not found: ID does not exist" containerID="b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088" Jul 06 19:51:35.025869 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.025834 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088"} err="failed to get container status \"b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088\": rpc error: code = NotFound desc = could not find container \"b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088\": container with ID starting with b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088 not found: ID does not exist" Jul 06 19:51:35.025869 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.025847 2581 scope.go:117] "RemoveContainer" containerID="bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c" Jul 06 19:51:35.026081 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:51:35.026065 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c\": container with ID starting with bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c not found: ID does not exist" containerID="bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c" Jul 06 19:51:35.026127 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.026083 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c"} err="failed to get container status \"bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c\": rpc error: code = NotFound desc = could not find container \"bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c\": container with ID starting with bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c not found: ID does not exist" Jul 06 19:51:35.026127 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.026094 2581 scope.go:117] "RemoveContainer" containerID="467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb" Jul 06 19:51:35.026274 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:51:35.026259 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb\": container with ID starting with 467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb not found: ID does not exist" containerID="467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb" Jul 06 19:51:35.026307 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.026278 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb"} err="failed to get container status \"467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb\": rpc error: code = NotFound desc = could not find container \"467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb\": container with ID starting with 467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb not found: ID does not exist" Jul 06 19:51:35.026307 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.026289 2581 scope.go:117] "RemoveContainer" containerID="af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e" Jul 06 19:51:35.026531 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:51:35.026515 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e\": container with ID starting with af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e not found: ID does not exist" containerID="af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e" Jul 06 19:51:35.026582 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.026533 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e"} err="failed to get container status \"af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e\": rpc error: code = NotFound desc = could not find container \"af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e\": container with ID starting with af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e not found: ID does not exist" Jul 06 19:51:35.026582 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.026544 2581 scope.go:117] "RemoveContainer" containerID="1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f" Jul 06 19:51:35.026745 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:51:35.026731 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f\": container with ID starting with 1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f not found: ID does not exist" containerID="1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f" Jul 06 19:51:35.026785 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.026747 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f"} err="failed to get container status \"1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f\": rpc error: code = NotFound desc = could not find container \"1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f\": container with ID starting with 1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f not found: ID does not exist" Jul 06 19:51:35.026785 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.026759 2581 scope.go:117] "RemoveContainer" containerID="254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f" Jul 06 19:51:35.026965 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.026947 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f"} err="failed to get container status \"254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f\": rpc error: code = NotFound desc = could not find container \"254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f\": container with ID starting with 254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f not found: ID does not exist" Jul 06 19:51:35.026965 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.026964 2581 scope.go:117] "RemoveContainer" containerID="f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76" Jul 06 19:51:35.027183 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.027164 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76"} err="failed to get container status \"f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76\": rpc error: code = NotFound desc = could not find container \"f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76\": container with ID starting with f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76 not found: ID does not exist" Jul 06 19:51:35.027233 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.027184 2581 scope.go:117] "RemoveContainer" containerID="b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088" Jul 06 19:51:35.027424 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.027409 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088"} err="failed to get container status \"b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088\": rpc error: code = NotFound desc = could not find container \"b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088\": container with ID starting with b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088 not found: ID does not exist" Jul 06 19:51:35.027474 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.027426 2581 scope.go:117] "RemoveContainer" containerID="bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c" Jul 06 19:51:35.027639 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.027621 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c"} err="failed to get container status \"bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c\": rpc error: code = NotFound desc = could not find container \"bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c\": container with ID starting with bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c not found: ID does not exist" Jul 06 19:51:35.027694 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.027639 2581 scope.go:117] "RemoveContainer" containerID="467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb" Jul 06 19:51:35.028027 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.027924 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb"} err="failed to get container status \"467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb\": rpc error: code = NotFound desc = could not find container \"467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb\": container with ID starting with 467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb not found: ID does not exist" Jul 06 19:51:35.028027 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.027949 2581 scope.go:117] "RemoveContainer" containerID="af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e" Jul 06 19:51:35.028290 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.028264 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e"} err="failed to get container status \"af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e\": rpc error: code = NotFound desc = could not find container \"af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e\": container with ID starting with af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e not found: ID does not exist" Jul 06 19:51:35.028290 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.028289 2581 scope.go:117] "RemoveContainer" containerID="1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f" Jul 06 19:51:35.028642 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.028613 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f"} err="failed to get container status \"1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f\": rpc error: code = NotFound desc = could not find container \"1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f\": container with ID starting with 1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f not found: ID does not exist" Jul 06 19:51:35.028642 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.028643 2581 scope.go:117] "RemoveContainer" containerID="254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f" Jul 06 19:51:35.029030 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.029010 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f"} err="failed to get container status \"254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f\": rpc error: code = NotFound desc = could not find container \"254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f\": container with ID starting with 254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f not found: ID does not exist" Jul 06 19:51:35.029030 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.029029 2581 scope.go:117] "RemoveContainer" containerID="f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76" Jul 06 19:51:35.029260 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.029236 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76"} err="failed to get container status \"f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76\": rpc error: code = NotFound desc = could not find container \"f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76\": container with ID starting with f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76 not found: ID does not exist" Jul 06 19:51:35.029307 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.029263 2581 scope.go:117] "RemoveContainer" containerID="b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088" Jul 06 19:51:35.029505 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.029487 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088"} err="failed to get container status \"b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088\": rpc error: code = NotFound desc = could not find container \"b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088\": container with ID starting with b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088 not found: ID does not exist" Jul 06 19:51:35.029553 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.029506 2581 scope.go:117] "RemoveContainer" containerID="bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c" Jul 06 19:51:35.029715 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.029698 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c"} err="failed to get container status \"bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c\": rpc error: code = NotFound desc = could not find container \"bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c\": container with ID starting with bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c not found: ID does not exist" Jul 06 19:51:35.029757 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.029716 2581 scope.go:117] "RemoveContainer" containerID="467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb" Jul 06 19:51:35.029950 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.029933 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:51:35.029987 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.029947 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb"} err="failed to get container status \"467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb\": rpc error: code = NotFound desc = could not find container \"467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb\": container with ID starting with 467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb not found: ID does not exist" Jul 06 19:51:35.029987 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.029962 2581 scope.go:117] "RemoveContainer" containerID="af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e" Jul 06 19:51:35.030171 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030157 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e"} err="failed to get container status \"af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e\": rpc error: code = NotFound desc = could not find container \"af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e\": container with ID starting with af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e not found: ID does not exist" Jul 06 19:51:35.030213 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030172 2581 scope.go:117] "RemoveContainer" containerID="1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f" Jul 06 19:51:35.030277 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030262 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="kube-rbac-proxy-web" Jul 06 19:51:35.030383 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030282 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="kube-rbac-proxy-web" Jul 06 19:51:35.030383 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030298 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="kube-rbac-proxy-thanos" Jul 06 19:51:35.030383 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030307 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="kube-rbac-proxy-thanos" Jul 06 19:51:35.030383 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030332 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="prometheus" Jul 06 19:51:35.030383 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030340 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="prometheus" Jul 06 19:51:35.030383 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030347 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="thanos-sidecar" Jul 06 19:51:35.030383 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030353 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="thanos-sidecar" Jul 06 19:51:35.030383 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030361 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="config-reloader" Jul 06 19:51:35.030383 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030366 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="config-reloader" Jul 06 19:51:35.030383 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030375 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="kube-rbac-proxy" Jul 06 19:51:35.030383 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030381 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="kube-rbac-proxy" Jul 06 19:51:35.030756 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030392 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="init-config-reloader" Jul 06 19:51:35.030756 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030397 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="init-config-reloader" Jul 06 19:51:35.030756 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030402 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f"} err="failed to get container status \"1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f\": rpc error: code = NotFound desc = could not find container \"1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f\": container with ID starting with 1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f not found: ID does not exist" Jul 06 19:51:35.030756 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030422 2581 scope.go:117] "RemoveContainer" containerID="254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f" Jul 06 19:51:35.030756 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030448 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="kube-rbac-proxy-thanos" Jul 06 19:51:35.030756 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030456 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="kube-rbac-proxy-web" Jul 06 19:51:35.030756 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030461 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="prometheus" Jul 06 19:51:35.030756 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030466 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="kube-rbac-proxy" Jul 06 19:51:35.030756 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030472 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="thanos-sidecar" Jul 06 19:51:35.030756 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030477 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" containerName="config-reloader" Jul 06 19:51:35.030756 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030651 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f"} err="failed to get container status \"254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f\": rpc error: code = NotFound desc = could not find container \"254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f\": container with ID starting with 254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f not found: ID does not exist" Jul 06 19:51:35.030756 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030667 2581 scope.go:117] "RemoveContainer" containerID="f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76" Jul 06 19:51:35.031176 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030874 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76"} err="failed to get container status \"f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76\": rpc error: code = NotFound desc = could not find container \"f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76\": container with ID starting with f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76 not found: ID does not exist" Jul 06 19:51:35.031176 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.030888 2581 scope.go:117] "RemoveContainer" containerID="b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088" Jul 06 19:51:35.031176 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.031103 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088"} err="failed to get container status \"b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088\": rpc error: code = NotFound desc = could not find container \"b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088\": container with ID starting with b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088 not found: ID does not exist" Jul 06 19:51:35.031176 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.031118 2581 scope.go:117] "RemoveContainer" containerID="bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c" Jul 06 19:51:35.031348 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.031329 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c"} err="failed to get container status \"bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c\": rpc error: code = NotFound desc = could not find container \"bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c\": container with ID starting with bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c not found: ID does not exist" Jul 06 19:51:35.031392 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.031350 2581 scope.go:117] "RemoveContainer" containerID="467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb" Jul 06 19:51:35.031583 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.031558 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb"} err="failed to get container status \"467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb\": rpc error: code = NotFound desc = could not find container \"467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb\": container with ID starting with 467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb not found: ID does not exist" Jul 06 19:51:35.031624 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.031583 2581 scope.go:117] "RemoveContainer" containerID="af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e" Jul 06 19:51:35.031788 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.031771 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e"} err="failed to get container status \"af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e\": rpc error: code = NotFound desc = could not find container \"af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e\": container with ID starting with af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e not found: ID does not exist" Jul 06 19:51:35.031824 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.031788 2581 scope.go:117] "RemoveContainer" containerID="1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f" Jul 06 19:51:35.031982 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.031966 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f"} err="failed to get container status \"1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f\": rpc error: code = NotFound desc = could not find container \"1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f\": container with ID starting with 1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f not found: ID does not exist" Jul 06 19:51:35.031982 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.031981 2581 scope.go:117] "RemoveContainer" containerID="254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f" Jul 06 19:51:35.032211 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.032193 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f"} err="failed to get container status \"254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f\": rpc error: code = NotFound desc = could not find container \"254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f\": container with ID starting with 254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f not found: ID does not exist" Jul 06 19:51:35.032258 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.032215 2581 scope.go:117] "RemoveContainer" containerID="f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76" Jul 06 19:51:35.032508 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.032487 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76"} err="failed to get container status \"f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76\": rpc error: code = NotFound desc = could not find container \"f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76\": container with ID starting with f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76 not found: ID does not exist" Jul 06 19:51:35.032579 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.032510 2581 scope.go:117] "RemoveContainer" containerID="b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088" Jul 06 19:51:35.032765 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.032742 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088"} err="failed to get container status \"b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088\": rpc error: code = NotFound desc = could not find container \"b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088\": container with ID starting with b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088 not found: ID does not exist" Jul 06 19:51:35.032765 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.032764 2581 scope.go:117] "RemoveContainer" containerID="bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c" Jul 06 19:51:35.033001 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.032974 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c"} err="failed to get container status \"bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c\": rpc error: code = NotFound desc = could not find container \"bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c\": container with ID starting with bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c not found: ID does not exist" Jul 06 19:51:35.033048 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.033002 2581 scope.go:117] "RemoveContainer" containerID="467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb" Jul 06 19:51:35.033218 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.033201 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb"} err="failed to get container status \"467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb\": rpc error: code = NotFound desc = could not find container \"467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb\": container with ID starting with 467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb not found: ID does not exist" Jul 06 19:51:35.033218 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.033217 2581 scope.go:117] "RemoveContainer" containerID="af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e" Jul 06 19:51:35.033467 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.033448 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e"} err="failed to get container status \"af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e\": rpc error: code = NotFound desc = could not find container \"af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e\": container with ID starting with af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e not found: ID does not exist" Jul 06 19:51:35.033519 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.033469 2581 scope.go:117] "RemoveContainer" containerID="1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f" Jul 06 19:51:35.033690 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.033674 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f"} err="failed to get container status \"1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f\": rpc error: code = NotFound desc = could not find container \"1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f\": container with ID starting with 1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f not found: ID does not exist" Jul 06 19:51:35.033728 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.033691 2581 scope.go:117] "RemoveContainer" containerID="254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f" Jul 06 19:51:35.033894 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.033878 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f"} err="failed to get container status \"254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f\": rpc error: code = NotFound desc = could not find container \"254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f\": container with ID starting with 254ec1158a3ce4ea7e7202e3fdfe4b043a63a54257bd1d73c5640922fe38a70f not found: ID does not exist" Jul 06 19:51:35.033894 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.033893 2581 scope.go:117] "RemoveContainer" containerID="f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76" Jul 06 19:51:35.034075 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.034059 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76"} err="failed to get container status \"f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76\": rpc error: code = NotFound desc = could not find container \"f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76\": container with ID starting with f8248f24447f7b4be581d229281f3e54a30ec16d8ee60e0dd8fa6e64a673db76 not found: ID does not exist" Jul 06 19:51:35.034118 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.034075 2581 scope.go:117] "RemoveContainer" containerID="b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088" Jul 06 19:51:35.034280 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.034264 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088"} err="failed to get container status \"b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088\": rpc error: code = NotFound desc = could not find container \"b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088\": container with ID starting with b4ba42b897f36cc627e713f7319b54efca79bc7e948494683c23804c6c3cf088 not found: ID does not exist" Jul 06 19:51:35.034351 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.034280 2581 scope.go:117] "RemoveContainer" containerID="bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c" Jul 06 19:51:35.034495 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.034480 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c"} err="failed to get container status \"bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c\": rpc error: code = NotFound desc = could not find container \"bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c\": container with ID starting with bdbd80cb7d1911e9ccb452700618ea828012f48986bbabc0a78f798a53d94a0c not found: ID does not exist" Jul 06 19:51:35.034538 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.034495 2581 scope.go:117] "RemoveContainer" containerID="467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb" Jul 06 19:51:35.034680 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.034666 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb"} err="failed to get container status \"467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb\": rpc error: code = NotFound desc = could not find container \"467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb\": container with ID starting with 467f4a7241466785f3791c52e81a6b56f7ae4a9695eee65b6dfdc3ba1ad1dfdb not found: ID does not exist" Jul 06 19:51:35.034726 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.034680 2581 scope.go:117] "RemoveContainer" containerID="af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e" Jul 06 19:51:35.034860 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.034845 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e"} err="failed to get container status \"af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e\": rpc error: code = NotFound desc = could not find container \"af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e\": container with ID starting with af466957bc440c98c4ed8c65ca8cdce94d5a2ddeddde6d443af0fd3591b5603e not found: ID does not exist" Jul 06 19:51:35.034908 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.034861 2581 scope.go:117] "RemoveContainer" containerID="1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f" Jul 06 19:51:35.035050 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.035035 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f"} err="failed to get container status \"1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f\": rpc error: code = NotFound desc = could not find container \"1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f\": container with ID starting with 1c2f343c9756ca38f98d142586837d687d43599a0ea91c4c804291fb63853a2f not found: ID does not exist" Jul 06 19:51:35.035972 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.035957 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.038613 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.038594 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 19:51:35.038705 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.038647 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6qtbb\"" Jul 06 19:51:35.038705 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.038694 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 19:51:35.038814 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.038783 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 19:51:35.038867 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.038830 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 19:51:35.038867 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.038835 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 19:51:35.039263 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.039250 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 19:51:35.039306 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.039280 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 19:51:35.039306 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.039294 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-84gg8b8u1nktj\"" Jul 06 19:51:35.039637 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.039623 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 19:51:35.039752 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.039653 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 19:51:35.039752 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.039653 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 19:51:35.039860 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.039758 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 19:51:35.040137 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.040123 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 19:51:35.040204 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.040139 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 19:51:35.041473 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.041449 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 19:51:35.045825 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.045808 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 19:51:35.052742 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.052723 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:51:35.094511 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.094482 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="458a7e6e-c625-401c-a9b2-75ea36214fe5" path="/var/lib/kubelet/pods/458a7e6e-c625-401c-a9b2-75ea36214fe5/volumes" Jul 06 19:51:35.175729 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.175653 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e38060bc-ae18-423c-9466-b62298ae81c4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.175729 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.175688 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e38060bc-ae18-423c-9466-b62298ae81c4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.175729 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.175707 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e38060bc-ae18-423c-9466-b62298ae81c4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.175916 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.175731 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e38060bc-ae18-423c-9466-b62298ae81c4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.175916 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.175765 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.175916 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.175787 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e38060bc-ae18-423c-9466-b62298ae81c4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.175916 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.175818 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.175916 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.175837 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e38060bc-ae18-423c-9466-b62298ae81c4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.175916 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.175857 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.175916 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.175883 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e38060bc-ae18-423c-9466-b62298ae81c4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.175916 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.175901 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mkhf\" (UniqueName: \"kubernetes.io/projected/e38060bc-ae18-423c-9466-b62298ae81c4-kube-api-access-4mkhf\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.176143 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.175931 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.176143 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.175969 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.176143 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.175990 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e38060bc-ae18-423c-9466-b62298ae81c4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.176143 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.176016 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e38060bc-ae18-423c-9466-b62298ae81c4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.176143 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.176079 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-web-config\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.176278 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.176172 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e38060bc-ae18-423c-9466-b62298ae81c4-config-out\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.176278 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.176222 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.176278 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.176242 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.176278 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.176258 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-config\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.277357 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.277300 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.277357 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.277359 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e38060bc-ae18-423c-9466-b62298ae81c4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.277576 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.277379 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mkhf\" (UniqueName: \"kubernetes.io/projected/e38060bc-ae18-423c-9466-b62298ae81c4-kube-api-access-4mkhf\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.277576 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.277399 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.277576 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.277420 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.277576 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.277436 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e38060bc-ae18-423c-9466-b62298ae81c4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.277576 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.277463 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e38060bc-ae18-423c-9466-b62298ae81c4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.277576 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.277537 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-web-config\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.277849 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.277587 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e38060bc-ae18-423c-9466-b62298ae81c4-config-out\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.277849 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.277640 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.277849 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.277674 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.277849 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.277700 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-config\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.277849 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.277736 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e38060bc-ae18-423c-9466-b62298ae81c4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.277849 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.277766 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e38060bc-ae18-423c-9466-b62298ae81c4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.277849 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.277796 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e38060bc-ae18-423c-9466-b62298ae81c4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.277849 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.277827 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e38060bc-ae18-423c-9466-b62298ae81c4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.278262 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.277854 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.278262 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.277891 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e38060bc-ae18-423c-9466-b62298ae81c4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.278262 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.277918 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.278262 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.277943 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e38060bc-ae18-423c-9466-b62298ae81c4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.278262 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.278236 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e38060bc-ae18-423c-9466-b62298ae81c4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.278817 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.278538 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e38060bc-ae18-423c-9466-b62298ae81c4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.280378 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.279057 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e38060bc-ae18-423c-9466-b62298ae81c4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.280378 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.279273 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e38060bc-ae18-423c-9466-b62298ae81c4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.280378 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.280290 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e38060bc-ae18-423c-9466-b62298ae81c4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.281588 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.281048 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.281588 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.281050 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.281588 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.281096 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.281588 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.281184 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e38060bc-ae18-423c-9466-b62298ae81c4-config-out\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.281588 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.281296 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.281588 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.281543 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.281588 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.281586 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e38060bc-ae18-423c-9466-b62298ae81c4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.281999 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.281694 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e38060bc-ae18-423c-9466-b62298ae81c4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.282150 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.282125 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e38060bc-ae18-423c-9466-b62298ae81c4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.282434 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.282407 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.282657 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.282639 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-web-config\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.282894 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.282875 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.283438 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.283416 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e38060bc-ae18-423c-9466-b62298ae81c4-config\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.283557 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.283538 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e38060bc-ae18-423c-9466-b62298ae81c4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.286197 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.286181 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mkhf\" (UniqueName: \"kubernetes.io/projected/e38060bc-ae18-423c-9466-b62298ae81c4-kube-api-access-4mkhf\") pod \"prometheus-k8s-0\" (UID: \"e38060bc-ae18-423c-9466-b62298ae81c4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.345857 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.345822 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:51:35.484598 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.484508 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 19:51:35.487464 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:51:35.487443 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode38060bc_ae18_423c_9466_b62298ae81c4.slice/crio-55575dc3335ae76bbc200f76db1a00e6fb00d4ade2522937450a60a3b4ad0b11 WatchSource:0}: Error finding container 55575dc3335ae76bbc200f76db1a00e6fb00d4ade2522937450a60a3b4ad0b11: Status 404 returned error can't find the container with id 55575dc3335ae76bbc200f76db1a00e6fb00d4ade2522937450a60a3b4ad0b11 Jul 06 19:51:35.982556 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.982525 2581 generic.go:356] "Generic (PLEG): container finished" podID="e38060bc-ae18-423c-9466-b62298ae81c4" containerID="63dcf5f288940bdae3c673f9d7b60103fe7abf95a57d26c540b54f4253c6bd4d" exitCode=0 Jul 06 19:51:35.982986 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.982575 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e38060bc-ae18-423c-9466-b62298ae81c4","Type":"ContainerDied","Data":"63dcf5f288940bdae3c673f9d7b60103fe7abf95a57d26c540b54f4253c6bd4d"} Jul 06 19:51:35.982986 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:35.982624 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e38060bc-ae18-423c-9466-b62298ae81c4","Type":"ContainerStarted","Data":"55575dc3335ae76bbc200f76db1a00e6fb00d4ade2522937450a60a3b4ad0b11"} Jul 06 19:51:36.992463 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:36.992432 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e38060bc-ae18-423c-9466-b62298ae81c4","Type":"ContainerStarted","Data":"fb7aa63152c6327b21505fa061b3665c5261e4794c06d637bd3a59007e0d2566"} Jul 06 19:51:36.992908 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:36.992493 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e38060bc-ae18-423c-9466-b62298ae81c4","Type":"ContainerStarted","Data":"32e8d7cd46c61d23d4a690b027de326e25931e9c8cf710ff943761e97b74336c"} Jul 06 19:51:36.992908 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:36.992521 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e38060bc-ae18-423c-9466-b62298ae81c4","Type":"ContainerStarted","Data":"7781947c33b999869267dd8da8cca625a5d1ebd19585ffbf5221318a82476825"} Jul 06 19:51:36.992908 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:36.992544 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e38060bc-ae18-423c-9466-b62298ae81c4","Type":"ContainerStarted","Data":"58cf254d1b942c177a4cd07987891986eefdc1a10df8dde9487b809aa0216061"} Jul 06 19:51:36.992908 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:36.992579 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e38060bc-ae18-423c-9466-b62298ae81c4","Type":"ContainerStarted","Data":"79dead9680a816a0d96db7cd5cb3a9223e640f623422bf2c5680ee7e0a69a94e"} Jul 06 19:51:36.992908 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:36.992616 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e38060bc-ae18-423c-9466-b62298ae81c4","Type":"ContainerStarted","Data":"990646b8f34ddeda8991a49257c44001853d547b7ce08fce8bc61492e9c19e32"} Jul 06 19:51:37.048626 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:37.048572 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.02865914 podStartE2EDuration="2.048554112s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.894972ms" isStatefulPod=false podCreationTimestamp="2026-07-06 19:51:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:51:37.040672589 +0000 UTC m=+196.487909332" watchObservedRunningTime="2026-07-06 19:51:37.048554112 +0000 UTC m=+196.495790829" Jul 06 19:51:40.346915 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:51:40.346880 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:52:35.346221 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:52:35.346182 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:52:35.361883 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:52:35.361859 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:52:36.231949 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:52:36.231923 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 19:53:02.031464 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:02.031418 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zcv8d"] Jul 06 19:53:02.034636 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:02.034620 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zcv8d" Jul 06 19:53:02.037090 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:02.037069 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:53:02.038165 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:02.038147 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-p24v8\"" Jul 06 19:53:02.038237 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:02.038148 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 19:53:02.043117 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:02.043092 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zcv8d"] Jul 06 19:53:02.128694 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:02.128661 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgbqn\" (UniqueName: \"kubernetes.io/projected/d988356c-af68-4828-be93-027302fdabb4-kube-api-access-vgbqn\") pod \"openshift-lws-operator-fd8ccff4c-zcv8d\" (UID: \"d988356c-af68-4828-be93-027302fdabb4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zcv8d" Jul 06 19:53:02.128694 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:02.128706 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d988356c-af68-4828-be93-027302fdabb4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zcv8d\" (UID: \"d988356c-af68-4828-be93-027302fdabb4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zcv8d" Jul 06 19:53:02.229508 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:02.229464 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgbqn\" (UniqueName: \"kubernetes.io/projected/d988356c-af68-4828-be93-027302fdabb4-kube-api-access-vgbqn\") pod \"openshift-lws-operator-fd8ccff4c-zcv8d\" (UID: \"d988356c-af68-4828-be93-027302fdabb4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zcv8d" Jul 06 19:53:02.229508 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:02.229509 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d988356c-af68-4828-be93-027302fdabb4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zcv8d\" (UID: \"d988356c-af68-4828-be93-027302fdabb4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zcv8d" Jul 06 19:53:02.229894 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:02.229872 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d988356c-af68-4828-be93-027302fdabb4-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zcv8d\" (UID: \"d988356c-af68-4828-be93-027302fdabb4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zcv8d" Jul 06 19:53:02.241767 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:02.241724 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgbqn\" (UniqueName: \"kubernetes.io/projected/d988356c-af68-4828-be93-027302fdabb4-kube-api-access-vgbqn\") pod \"openshift-lws-operator-fd8ccff4c-zcv8d\" (UID: \"d988356c-af68-4828-be93-027302fdabb4\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zcv8d" Jul 06 19:53:02.357921 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:02.357837 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zcv8d" Jul 06 19:53:02.483824 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:02.483800 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zcv8d"] Jul 06 19:53:02.485734 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:53:02.485710 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd988356c_af68_4828_be93_027302fdabb4.slice/crio-11ab9eee1bf55dc793533de8775ba9e8f6c7403eb1b21c9430a445910b8abb35 WatchSource:0}: Error finding container 11ab9eee1bf55dc793533de8775ba9e8f6c7403eb1b21c9430a445910b8abb35: Status 404 returned error can't find the container with id 11ab9eee1bf55dc793533de8775ba9e8f6c7403eb1b21c9430a445910b8abb35 Jul 06 19:53:03.322796 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:03.322750 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zcv8d" event={"ID":"d988356c-af68-4828-be93-027302fdabb4","Type":"ContainerStarted","Data":"11ab9eee1bf55dc793533de8775ba9e8f6c7403eb1b21c9430a445910b8abb35"} Jul 06 19:53:07.345972 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:07.345934 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zcv8d" event={"ID":"d988356c-af68-4828-be93-027302fdabb4","Type":"ContainerStarted","Data":"a8859b6c946b5837dda08cce2780cc2c4a07b76f495a77f52602830ea3ab836e"} Jul 06 19:53:07.370379 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:07.370311 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zcv8d" podStartSLOduration=1.133371391 podStartE2EDuration="5.370295516s" totalImagesPullingTime="4.236924125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:53:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:53:07.369047103 +0000 UTC m=+286.816283823" watchObservedRunningTime="2026-07-06 19:53:07.370295516 +0000 UTC m=+286.817532233" Jul 06 19:53:20.979450 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:20.979424 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/2.log" Jul 06 19:53:20.983969 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:20.983948 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/2.log" Jul 06 19:53:20.988567 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:20.988545 2581 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 19:53:22.725985 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.725953 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm"] Jul 06 19:53:22.729499 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.729483 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" Jul 06 19:53:22.732381 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.732362 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 19:53:22.732381 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.732384 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 19:53:22.732560 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.732417 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6l89k\"" Jul 06 19:53:22.732560 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.732449 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 19:53:22.732560 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.732367 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 19:53:22.744196 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.744177 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm"] Jul 06 19:53:22.804331 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.804277 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs2zq\" (UniqueName: \"kubernetes.io/projected/f412df03-f5f8-4647-bd8e-beb4c9e6a60f-kube-api-access-rs2zq\") pod \"opendatahub-operator-controller-manager-74dff95f86-gnhkm\" (UID: \"f412df03-f5f8-4647-bd8e-beb4c9e6a60f\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" Jul 06 19:53:22.804497 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.804343 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f412df03-f5f8-4647-bd8e-beb4c9e6a60f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74dff95f86-gnhkm\" (UID: \"f412df03-f5f8-4647-bd8e-beb4c9e6a60f\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" Jul 06 19:53:22.804497 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.804369 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f412df03-f5f8-4647-bd8e-beb4c9e6a60f-webhook-cert\") pod \"opendatahub-operator-controller-manager-74dff95f86-gnhkm\" (UID: \"f412df03-f5f8-4647-bd8e-beb4c9e6a60f\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" Jul 06 19:53:22.804497 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.804396 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f412df03-f5f8-4647-bd8e-beb4c9e6a60f-tmp\") pod \"opendatahub-operator-controller-manager-74dff95f86-gnhkm\" (UID: \"f412df03-f5f8-4647-bd8e-beb4c9e6a60f\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" Jul 06 19:53:22.804497 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.804432 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f412df03-f5f8-4647-bd8e-beb4c9e6a60f-manifests\") pod \"opendatahub-operator-controller-manager-74dff95f86-gnhkm\" (UID: \"f412df03-f5f8-4647-bd8e-beb4c9e6a60f\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" Jul 06 19:53:22.905017 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.904981 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs2zq\" (UniqueName: \"kubernetes.io/projected/f412df03-f5f8-4647-bd8e-beb4c9e6a60f-kube-api-access-rs2zq\") pod \"opendatahub-operator-controller-manager-74dff95f86-gnhkm\" (UID: \"f412df03-f5f8-4647-bd8e-beb4c9e6a60f\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" Jul 06 19:53:22.905017 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.905020 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f412df03-f5f8-4647-bd8e-beb4c9e6a60f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74dff95f86-gnhkm\" (UID: \"f412df03-f5f8-4647-bd8e-beb4c9e6a60f\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" Jul 06 19:53:22.905235 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.905043 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f412df03-f5f8-4647-bd8e-beb4c9e6a60f-webhook-cert\") pod \"opendatahub-operator-controller-manager-74dff95f86-gnhkm\" (UID: \"f412df03-f5f8-4647-bd8e-beb4c9e6a60f\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" Jul 06 19:53:22.905235 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.905080 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f412df03-f5f8-4647-bd8e-beb4c9e6a60f-tmp\") pod \"opendatahub-operator-controller-manager-74dff95f86-gnhkm\" (UID: \"f412df03-f5f8-4647-bd8e-beb4c9e6a60f\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" Jul 06 19:53:22.905235 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.905130 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f412df03-f5f8-4647-bd8e-beb4c9e6a60f-manifests\") pod \"opendatahub-operator-controller-manager-74dff95f86-gnhkm\" (UID: \"f412df03-f5f8-4647-bd8e-beb4c9e6a60f\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" Jul 06 19:53:22.905533 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.905512 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f412df03-f5f8-4647-bd8e-beb4c9e6a60f-manifests\") pod \"opendatahub-operator-controller-manager-74dff95f86-gnhkm\" (UID: \"f412df03-f5f8-4647-bd8e-beb4c9e6a60f\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" Jul 06 19:53:22.907242 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.907216 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f412df03-f5f8-4647-bd8e-beb4c9e6a60f-tmp\") pod \"opendatahub-operator-controller-manager-74dff95f86-gnhkm\" (UID: \"f412df03-f5f8-4647-bd8e-beb4c9e6a60f\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" Jul 06 19:53:22.907600 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.907578 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f412df03-f5f8-4647-bd8e-beb4c9e6a60f-webhook-cert\") pod \"opendatahub-operator-controller-manager-74dff95f86-gnhkm\" (UID: \"f412df03-f5f8-4647-bd8e-beb4c9e6a60f\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" Jul 06 19:53:22.907695 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.907680 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f412df03-f5f8-4647-bd8e-beb4c9e6a60f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-74dff95f86-gnhkm\" (UID: \"f412df03-f5f8-4647-bd8e-beb4c9e6a60f\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" Jul 06 19:53:22.915887 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:22.915862 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs2zq\" (UniqueName: \"kubernetes.io/projected/f412df03-f5f8-4647-bd8e-beb4c9e6a60f-kube-api-access-rs2zq\") pod \"opendatahub-operator-controller-manager-74dff95f86-gnhkm\" (UID: \"f412df03-f5f8-4647-bd8e-beb4c9e6a60f\") " pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" Jul 06 19:53:23.039223 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:23.039189 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" Jul 06 19:53:23.182772 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:23.182734 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm"] Jul 06 19:53:23.187986 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:53:23.187958 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf412df03_f5f8_4647_bd8e_beb4c9e6a60f.slice/crio-38abb790ff0b51d2a211245b19c67ee09068ad28a77079ae2723dfcd50fa4967 WatchSource:0}: Error finding container 38abb790ff0b51d2a211245b19c67ee09068ad28a77079ae2723dfcd50fa4967: Status 404 returned error can't find the container with id 38abb790ff0b51d2a211245b19c67ee09068ad28a77079ae2723dfcd50fa4967 Jul 06 19:53:23.190034 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:23.190018 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:53:23.417244 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:23.417157 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" event={"ID":"f412df03-f5f8-4647-bd8e-beb4c9e6a60f","Type":"ContainerStarted","Data":"38abb790ff0b51d2a211245b19c67ee09068ad28a77079ae2723dfcd50fa4967"} Jul 06 19:53:26.436624 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:26.436587 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" event={"ID":"f412df03-f5f8-4647-bd8e-beb4c9e6a60f","Type":"ContainerStarted","Data":"573f9fa777c49e6de4ca92f69aff0e1f7fea39370322825d902f9f89c7df65c6"} Jul 06 19:53:28.446621 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:28.446587 2581 generic.go:356] "Generic (PLEG): container finished" podID="f412df03-f5f8-4647-bd8e-beb4c9e6a60f" containerID="573f9fa777c49e6de4ca92f69aff0e1f7fea39370322825d902f9f89c7df65c6" exitCode=0 Jul 06 19:53:28.447075 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:28.446668 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" event={"ID":"f412df03-f5f8-4647-bd8e-beb4c9e6a60f","Type":"ContainerDied","Data":"573f9fa777c49e6de4ca92f69aff0e1f7fea39370322825d902f9f89c7df65c6"} Jul 06 19:53:29.452811 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:29.452782 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" event={"ID":"f412df03-f5f8-4647-bd8e-beb4c9e6a60f","Type":"ContainerStarted","Data":"fc1eaf2079744ad06c8574eab92ede86b3afb99b87c1148437339ba19a5731a2"} Jul 06 19:53:29.453193 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:29.452998 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" Jul 06 19:53:29.493677 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:29.493626 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" podStartSLOduration=2.992472596 podStartE2EDuration="7.493607325s" totalImagesPullingTime="3.476399966s" totalInitContainerRuntime="1.024734763s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:53:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:53:29.49120712 +0000 UTC m=+308.938443838" watchObservedRunningTime="2026-07-06 19:53:29.493607325 +0000 UTC m=+308.940844043" Jul 06 19:53:40.459942 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:40.459911 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-74dff95f86-gnhkm" Jul 06 19:53:51.644783 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:51.644747 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-59cb764c9c-qvj5k"] Jul 06 19:53:51.648060 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:51.648039 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59cb764c9c-qvj5k" Jul 06 19:53:51.651748 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:51.651724 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 19:53:51.651855 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:51.651764 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 19:53:51.651855 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:51.651790 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 19:53:51.651855 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:51.651837 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-gsgts\"" Jul 06 19:53:51.660123 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:51.660102 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59cb764c9c-qvj5k"] Jul 06 19:53:51.768133 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:51.768107 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1-metrics-cert\") pod \"lws-controller-manager-59cb764c9c-qvj5k\" (UID: \"2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1\") " pod="openshift-lws-operator/lws-controller-manager-59cb764c9c-qvj5k" Jul 06 19:53:51.768332 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:51.768139 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1-cert\") pod \"lws-controller-manager-59cb764c9c-qvj5k\" (UID: \"2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1\") " pod="openshift-lws-operator/lws-controller-manager-59cb764c9c-qvj5k" Jul 06 19:53:51.768332 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:51.768164 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8tj7\" (UniqueName: \"kubernetes.io/projected/2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1-kube-api-access-s8tj7\") pod \"lws-controller-manager-59cb764c9c-qvj5k\" (UID: \"2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1\") " pod="openshift-lws-operator/lws-controller-manager-59cb764c9c-qvj5k" Jul 06 19:53:51.768332 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:51.768289 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1-manager-config\") pod \"lws-controller-manager-59cb764c9c-qvj5k\" (UID: \"2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1\") " pod="openshift-lws-operator/lws-controller-manager-59cb764c9c-qvj5k" Jul 06 19:53:51.869073 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:51.869031 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1-metrics-cert\") pod \"lws-controller-manager-59cb764c9c-qvj5k\" (UID: \"2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1\") " pod="openshift-lws-operator/lws-controller-manager-59cb764c9c-qvj5k" Jul 06 19:53:51.869073 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:51.869080 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1-cert\") pod \"lws-controller-manager-59cb764c9c-qvj5k\" (UID: \"2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1\") " pod="openshift-lws-operator/lws-controller-manager-59cb764c9c-qvj5k" Jul 06 19:53:51.869312 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:51.869128 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8tj7\" (UniqueName: \"kubernetes.io/projected/2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1-kube-api-access-s8tj7\") pod \"lws-controller-manager-59cb764c9c-qvj5k\" (UID: \"2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1\") " pod="openshift-lws-operator/lws-controller-manager-59cb764c9c-qvj5k" Jul 06 19:53:51.869312 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:51.869205 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1-manager-config\") pod \"lws-controller-manager-59cb764c9c-qvj5k\" (UID: \"2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1\") " pod="openshift-lws-operator/lws-controller-manager-59cb764c9c-qvj5k" Jul 06 19:53:51.869921 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:51.869884 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1-manager-config\") pod \"lws-controller-manager-59cb764c9c-qvj5k\" (UID: \"2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1\") " pod="openshift-lws-operator/lws-controller-manager-59cb764c9c-qvj5k" Jul 06 19:53:51.871646 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:51.871622 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1-metrics-cert\") pod \"lws-controller-manager-59cb764c9c-qvj5k\" (UID: \"2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1\") " pod="openshift-lws-operator/lws-controller-manager-59cb764c9c-qvj5k" Jul 06 19:53:51.872078 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:51.872041 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1-cert\") pod \"lws-controller-manager-59cb764c9c-qvj5k\" (UID: \"2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1\") " pod="openshift-lws-operator/lws-controller-manager-59cb764c9c-qvj5k" Jul 06 19:53:51.878552 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:51.878526 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8tj7\" (UniqueName: \"kubernetes.io/projected/2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1-kube-api-access-s8tj7\") pod \"lws-controller-manager-59cb764c9c-qvj5k\" (UID: \"2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1\") " pod="openshift-lws-operator/lws-controller-manager-59cb764c9c-qvj5k" Jul 06 19:53:51.957832 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:51.957718 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59cb764c9c-qvj5k" Jul 06 19:53:52.122049 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:52.122020 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59cb764c9c-qvj5k"] Jul 06 19:53:52.123589 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:53:52.123561 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f04ef4f_fea0_4bf7_871d_5162dfcd6cd1.slice/crio-b465d81a452e2418474490947686685af19921020991c7df0c63473466da5643 WatchSource:0}: Error finding container b465d81a452e2418474490947686685af19921020991c7df0c63473466da5643: Status 404 returned error can't find the container with id b465d81a452e2418474490947686685af19921020991c7df0c63473466da5643 Jul 06 19:53:52.550524 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:52.550486 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59cb764c9c-qvj5k" event={"ID":"2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1","Type":"ContainerStarted","Data":"b465d81a452e2418474490947686685af19921020991c7df0c63473466da5643"} Jul 06 19:53:53.131381 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.131351 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-f2849"] Jul 06 19:53:53.135625 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.135606 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2849" Jul 06 19:53:53.138303 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.138282 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 06 19:53:53.139396 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.139376 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-kvbqq\"" Jul 06 19:53:53.153111 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.153088 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-f2849"] Jul 06 19:53:53.167445 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.167424 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-t9ntm"] Jul 06 19:53:53.170552 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.170535 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t9ntm" Jul 06 19:53:53.173330 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.173297 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 06 19:53:53.173417 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.173368 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-x6f82\"" Jul 06 19:53:53.183632 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.183603 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hmnn\" (UniqueName: \"kubernetes.io/projected/442fc246-0e4b-40cb-9cb9-4d11286a5676-kube-api-access-2hmnn\") pod \"kserve-controller-manager-d6f4b65b8-f2849\" (UID: \"442fc246-0e4b-40cb-9cb9-4d11286a5676\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2849" Jul 06 19:53:53.183784 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.183764 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/442fc246-0e4b-40cb-9cb9-4d11286a5676-cert\") pod \"kserve-controller-manager-d6f4b65b8-f2849\" (UID: \"442fc246-0e4b-40cb-9cb9-4d11286a5676\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2849" Jul 06 19:53:53.184093 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.184070 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-t9ntm"] Jul 06 19:53:53.285152 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.285113 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff9f8daa-34bd-47f2-84ab-818793008a03-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-t9ntm\" (UID: \"ff9f8daa-34bd-47f2-84ab-818793008a03\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t9ntm" Jul 06 19:53:53.285316 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.285178 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/442fc246-0e4b-40cb-9cb9-4d11286a5676-cert\") pod \"kserve-controller-manager-d6f4b65b8-f2849\" (UID: \"442fc246-0e4b-40cb-9cb9-4d11286a5676\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2849" Jul 06 19:53:53.285316 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.285227 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hmnn\" (UniqueName: \"kubernetes.io/projected/442fc246-0e4b-40cb-9cb9-4d11286a5676-kube-api-access-2hmnn\") pod \"kserve-controller-manager-d6f4b65b8-f2849\" (UID: \"442fc246-0e4b-40cb-9cb9-4d11286a5676\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2849" Jul 06 19:53:53.285316 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.285258 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsq6m\" (UniqueName: \"kubernetes.io/projected/ff9f8daa-34bd-47f2-84ab-818793008a03-kube-api-access-fsq6m\") pod \"llmisvc-controller-manager-777c6bd7d5-t9ntm\" (UID: \"ff9f8daa-34bd-47f2-84ab-818793008a03\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t9ntm" Jul 06 19:53:53.287834 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.287803 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/442fc246-0e4b-40cb-9cb9-4d11286a5676-cert\") pod \"kserve-controller-manager-d6f4b65b8-f2849\" (UID: \"442fc246-0e4b-40cb-9cb9-4d11286a5676\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2849" Jul 06 19:53:53.293774 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.293743 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hmnn\" (UniqueName: \"kubernetes.io/projected/442fc246-0e4b-40cb-9cb9-4d11286a5676-kube-api-access-2hmnn\") pod \"kserve-controller-manager-d6f4b65b8-f2849\" (UID: \"442fc246-0e4b-40cb-9cb9-4d11286a5676\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2849" Jul 06 19:53:53.386204 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.386114 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsq6m\" (UniqueName: \"kubernetes.io/projected/ff9f8daa-34bd-47f2-84ab-818793008a03-kube-api-access-fsq6m\") pod \"llmisvc-controller-manager-777c6bd7d5-t9ntm\" (UID: \"ff9f8daa-34bd-47f2-84ab-818793008a03\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t9ntm" Jul 06 19:53:53.386204 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.386165 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff9f8daa-34bd-47f2-84ab-818793008a03-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-t9ntm\" (UID: \"ff9f8daa-34bd-47f2-84ab-818793008a03\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t9ntm" Jul 06 19:53:53.388685 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.388661 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ff9f8daa-34bd-47f2-84ab-818793008a03-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-t9ntm\" (UID: \"ff9f8daa-34bd-47f2-84ab-818793008a03\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t9ntm" Jul 06 19:53:53.405675 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.405647 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsq6m\" (UniqueName: \"kubernetes.io/projected/ff9f8daa-34bd-47f2-84ab-818793008a03-kube-api-access-fsq6m\") pod \"llmisvc-controller-manager-777c6bd7d5-t9ntm\" (UID: \"ff9f8daa-34bd-47f2-84ab-818793008a03\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t9ntm" Jul 06 19:53:53.446125 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.446088 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2849" Jul 06 19:53:53.480461 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.480429 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t9ntm" Jul 06 19:53:53.598263 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.598094 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-f2849"] Jul 06 19:53:53.601476 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:53:53.601444 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod442fc246_0e4b_40cb_9cb9_4d11286a5676.slice/crio-d223f555f4db5a813f167a508d137d69f44328cb344a6212e1c8d9cc7e764f1e WatchSource:0}: Error finding container d223f555f4db5a813f167a508d137d69f44328cb344a6212e1c8d9cc7e764f1e: Status 404 returned error can't find the container with id d223f555f4db5a813f167a508d137d69f44328cb344a6212e1c8d9cc7e764f1e Jul 06 19:53:53.638160 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:53.638063 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-t9ntm"] Jul 06 19:53:53.641562 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:53:53.641534 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podff9f8daa_34bd_47f2_84ab_818793008a03.slice/crio-92ad466064453f40e29489d1d09f11ffe03b5349194a278ed523cc93394a73fd WatchSource:0}: Error finding container 92ad466064453f40e29489d1d09f11ffe03b5349194a278ed523cc93394a73fd: Status 404 returned error can't find the container with id 92ad466064453f40e29489d1d09f11ffe03b5349194a278ed523cc93394a73fd Jul 06 19:53:54.544825 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.544788 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6"] Jul 06 19:53:54.548205 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.548182 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.553721 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.553700 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 19:53:54.553839 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.553701 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 19:53:54.553985 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.553967 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-6wpkb\"" Jul 06 19:53:54.554080 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.554065 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 19:53:54.564015 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.563982 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t9ntm" event={"ID":"ff9f8daa-34bd-47f2-84ab-818793008a03","Type":"ContainerStarted","Data":"92ad466064453f40e29489d1d09f11ffe03b5349194a278ed523cc93394a73fd"} Jul 06 19:53:54.565222 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.565188 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2849" event={"ID":"442fc246-0e4b-40cb-9cb9-4d11286a5676","Type":"ContainerStarted","Data":"d223f555f4db5a813f167a508d137d69f44328cb344a6212e1c8d9cc7e764f1e"} Jul 06 19:53:54.579707 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.579675 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6"] Jul 06 19:53:54.596811 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.596781 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2472da42-8346-454f-ae81-7cdea6bbb2c5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.596986 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.596838 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2472da42-8346-454f-ae81-7cdea6bbb2c5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.596986 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.596915 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2472da42-8346-454f-ae81-7cdea6bbb2c5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.596986 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.596939 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2472da42-8346-454f-ae81-7cdea6bbb2c5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.596986 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.596977 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2472da42-8346-454f-ae81-7cdea6bbb2c5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.597193 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.597017 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2472da42-8346-454f-ae81-7cdea6bbb2c5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.597193 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.597038 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9s8sj\" (UniqueName: \"kubernetes.io/projected/2472da42-8346-454f-ae81-7cdea6bbb2c5-kube-api-access-9s8sj\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.597193 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.597081 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2472da42-8346-454f-ae81-7cdea6bbb2c5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.597193 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.597096 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2472da42-8346-454f-ae81-7cdea6bbb2c5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.697916 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.697880 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2472da42-8346-454f-ae81-7cdea6bbb2c5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.698091 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.697922 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2472da42-8346-454f-ae81-7cdea6bbb2c5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.698091 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.698042 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2472da42-8346-454f-ae81-7cdea6bbb2c5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.698207 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.698106 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2472da42-8346-454f-ae81-7cdea6bbb2c5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.698207 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.698131 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9s8sj\" (UniqueName: \"kubernetes.io/projected/2472da42-8346-454f-ae81-7cdea6bbb2c5-kube-api-access-9s8sj\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.698207 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.698200 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2472da42-8346-454f-ae81-7cdea6bbb2c5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.698420 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.698229 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2472da42-8346-454f-ae81-7cdea6bbb2c5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.698420 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.698273 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2472da42-8346-454f-ae81-7cdea6bbb2c5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.698420 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.698303 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2472da42-8346-454f-ae81-7cdea6bbb2c5-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.698420 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.698341 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2472da42-8346-454f-ae81-7cdea6bbb2c5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.698420 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.698374 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2472da42-8346-454f-ae81-7cdea6bbb2c5-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.698670 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.698574 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2472da42-8346-454f-ae81-7cdea6bbb2c5-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.698720 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.698679 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2472da42-8346-454f-ae81-7cdea6bbb2c5-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.698816 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.698777 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2472da42-8346-454f-ae81-7cdea6bbb2c5-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.700491 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.700461 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2472da42-8346-454f-ae81-7cdea6bbb2c5-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.701109 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.701088 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2472da42-8346-454f-ae81-7cdea6bbb2c5-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.714825 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.714796 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9s8sj\" (UniqueName: \"kubernetes.io/projected/2472da42-8346-454f-ae81-7cdea6bbb2c5-kube-api-access-9s8sj\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.717003 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.716982 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2472da42-8346-454f-ae81-7cdea6bbb2c5-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4hlkx6\" (UID: \"2472da42-8346-454f-ae81-7cdea6bbb2c5\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:54.862522 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:54.861903 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:53:55.243036 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:55.242951 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6"] Jul 06 19:53:55.245075 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:53:55.245043 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2472da42_8346_454f_ae81_7cdea6bbb2c5.slice/crio-042a980654e625b7b566edd6801420664e2df94758a849ecb755cede893707d9 WatchSource:0}: Error finding container 042a980654e625b7b566edd6801420664e2df94758a849ecb755cede893707d9: Status 404 returned error can't find the container with id 042a980654e625b7b566edd6801420664e2df94758a849ecb755cede893707d9 Jul 06 19:53:55.572338 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:55.572283 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" event={"ID":"2472da42-8346-454f-ae81-7cdea6bbb2c5","Type":"ContainerStarted","Data":"042a980654e625b7b566edd6801420664e2df94758a849ecb755cede893707d9"} Jul 06 19:53:58.597259 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:58.597171 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t9ntm" event={"ID":"ff9f8daa-34bd-47f2-84ab-818793008a03","Type":"ContainerStarted","Data":"8a79c770b0b784f4c14040927be10e8feec8715d848fc4bcb29395de9e9376b8"} Jul 06 19:53:58.597760 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:58.597446 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t9ntm" Jul 06 19:53:58.599036 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:58.599009 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2849" event={"ID":"442fc246-0e4b-40cb-9cb9-4d11286a5676","Type":"ContainerStarted","Data":"6733ba0998ed64d2a8f78f2f5d71cad24eaa684577a28045e0983b88397301e9"} Jul 06 19:53:58.599167 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:58.599152 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2849" Jul 06 19:53:58.615642 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:58.615587 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t9ntm" podStartSLOduration=1.702384083 podStartE2EDuration="5.615571718s" totalImagesPullingTime="3.913187635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:53:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:53:58.612981765 +0000 UTC m=+338.060218484" watchObservedRunningTime="2026-07-06 19:53:58.615571718 +0000 UTC m=+338.062808437" Jul 06 19:53:58.630355 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:58.630245 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2849" podStartSLOduration=1.681432523 podStartE2EDuration="5.630226807s" totalImagesPullingTime="3.948794284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:53:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:53:58.629514922 +0000 UTC m=+338.076751642" watchObservedRunningTime="2026-07-06 19:53:58.630226807 +0000 UTC m=+338.077463529" Jul 06 19:53:58.938771 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:58.938740 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 19:53:58.938846 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:58.938809 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 19:53:58.938846 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:58.938839 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 19:53:59.604907 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:59.604869 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" event={"ID":"2472da42-8346-454f-ae81-7cdea6bbb2c5","Type":"ContainerStarted","Data":"5e236d60441e601f0a525a9452e791c85ac0e376848e9a5b1e67cbecb2c00531"} Jul 06 19:53:59.630593 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:59.630543 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" podStartSLOduration=1.9396033419999998 podStartE2EDuration="5.630527211s" totalImagesPullingTime="3.690923869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:53:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:53:59.627894781 +0000 UTC m=+339.075131499" watchObservedRunningTime="2026-07-06 19:53:59.630527211 +0000 UTC m=+339.077763929" Jul 06 19:53:59.862607 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:53:59.862527 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:54:00.867723 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:54:00.867692 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:54:01.613293 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:54:01.613261 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:54:01.614277 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:54:01.614260 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4hlkx6" Jul 06 19:54:14.671939 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:54:14.671906 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59cb764c9c-qvj5k" event={"ID":"2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1","Type":"ContainerStarted","Data":"33d71da0c9c229288c937f188cb30fcec1d577d5e99e609196ab2646371aaa04"} Jul 06 19:54:14.672363 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:54:14.671987 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-59cb764c9c-qvj5k" Jul 06 19:54:14.691290 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:54:14.691239 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-59cb764c9c-qvj5k" podStartSLOduration=2.133054077 podStartE2EDuration="23.691224432s" totalImagesPullingTime="21.558170355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:53:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:54:14.690088317 +0000 UTC m=+354.137325037" watchObservedRunningTime="2026-07-06 19:54:14.691224432 +0000 UTC m=+354.138461149" Jul 06 19:54:25.679073 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:54:25.679038 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-59cb764c9c-qvj5k" Jul 06 19:54:29.607926 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:54:29.607896 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-t9ntm" Jul 06 19:54:29.610940 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:54:29.610917 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f2849" Jul 06 19:55:45.865169 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:45.865087 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sl8vx"] Jul 06 19:55:45.867508 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:45.867490 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sl8vx" Jul 06 19:55:45.870280 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:45.870261 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 19:55:45.870415 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:45.870262 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tckhr\"" Jul 06 19:55:45.870761 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:45.870743 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 19:55:45.883015 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:45.882985 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sl8vx"] Jul 06 19:55:45.979416 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:45.979377 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e6cf12a-4d1d-433f-93a1-c073560103a9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-sl8vx\" (UID: \"8e6cf12a-4d1d-433f-93a1-c073560103a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sl8vx" Jul 06 19:55:45.979595 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:45.979436 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz6sz\" (UniqueName: \"kubernetes.io/projected/8e6cf12a-4d1d-433f-93a1-c073560103a9-kube-api-access-rz6sz\") pod \"kuadrant-operator-controller-manager-55467d9f7-sl8vx\" (UID: \"8e6cf12a-4d1d-433f-93a1-c073560103a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sl8vx" Jul 06 19:55:46.080652 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:46.080612 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e6cf12a-4d1d-433f-93a1-c073560103a9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-sl8vx\" (UID: \"8e6cf12a-4d1d-433f-93a1-c073560103a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sl8vx" Jul 06 19:55:46.080792 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:46.080669 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rz6sz\" (UniqueName: \"kubernetes.io/projected/8e6cf12a-4d1d-433f-93a1-c073560103a9-kube-api-access-rz6sz\") pod \"kuadrant-operator-controller-manager-55467d9f7-sl8vx\" (UID: \"8e6cf12a-4d1d-433f-93a1-c073560103a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sl8vx" Jul 06 19:55:46.081012 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:46.080993 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e6cf12a-4d1d-433f-93a1-c073560103a9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-sl8vx\" (UID: \"8e6cf12a-4d1d-433f-93a1-c073560103a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sl8vx" Jul 06 19:55:46.089025 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:46.088993 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz6sz\" (UniqueName: \"kubernetes.io/projected/8e6cf12a-4d1d-433f-93a1-c073560103a9-kube-api-access-rz6sz\") pod \"kuadrant-operator-controller-manager-55467d9f7-sl8vx\" (UID: \"8e6cf12a-4d1d-433f-93a1-c073560103a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sl8vx" Jul 06 19:55:46.178628 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:46.178542 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sl8vx" Jul 06 19:55:46.317192 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:46.317116 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sl8vx"] Jul 06 19:55:46.319669 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:55:46.319638 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e6cf12a_4d1d_433f_93a1_c073560103a9.slice/crio-88027bfd4ed64fc055c051e204a3f9dbc115cb81aedd789fab052fa9c0f42579 WatchSource:0}: Error finding container 88027bfd4ed64fc055c051e204a3f9dbc115cb81aedd789fab052fa9c0f42579: Status 404 returned error can't find the container with id 88027bfd4ed64fc055c051e204a3f9dbc115cb81aedd789fab052fa9c0f42579 Jul 06 19:55:46.425959 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:46.425923 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sl8vx"] Jul 06 19:55:46.439274 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:46.439188 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sl8vx"] Jul 06 19:55:46.452923 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:46.452875 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn"] Jul 06 19:55:46.457308 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:46.457280 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn" Jul 06 19:55:46.474992 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:46.474960 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn"] Jul 06 19:55:46.585991 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:46.585954 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnjkv\" (UniqueName: \"kubernetes.io/projected/c78ef021-0d64-4e88-854b-036f17dd2401-kube-api-access-vnjkv\") pod \"kuadrant-operator-controller-manager-55467d9f7-87hgn\" (UID: \"c78ef021-0d64-4e88-854b-036f17dd2401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn" Jul 06 19:55:46.586212 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:46.586003 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c78ef021-0d64-4e88-854b-036f17dd2401-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-87hgn\" (UID: \"c78ef021-0d64-4e88-854b-036f17dd2401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn" Jul 06 19:55:46.686854 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:46.686814 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnjkv\" (UniqueName: \"kubernetes.io/projected/c78ef021-0d64-4e88-854b-036f17dd2401-kube-api-access-vnjkv\") pod \"kuadrant-operator-controller-manager-55467d9f7-87hgn\" (UID: \"c78ef021-0d64-4e88-854b-036f17dd2401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn" Jul 06 19:55:46.687026 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:46.686869 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c78ef021-0d64-4e88-854b-036f17dd2401-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-87hgn\" (UID: \"c78ef021-0d64-4e88-854b-036f17dd2401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn" Jul 06 19:55:46.687243 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:46.687225 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c78ef021-0d64-4e88-854b-036f17dd2401-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-87hgn\" (UID: \"c78ef021-0d64-4e88-854b-036f17dd2401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn" Jul 06 19:55:46.696708 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:46.696647 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnjkv\" (UniqueName: \"kubernetes.io/projected/c78ef021-0d64-4e88-854b-036f17dd2401-kube-api-access-vnjkv\") pod \"kuadrant-operator-controller-manager-55467d9f7-87hgn\" (UID: \"c78ef021-0d64-4e88-854b-036f17dd2401\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn" Jul 06 19:55:46.770778 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:46.770738 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn" Jul 06 19:55:46.911126 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:46.911022 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn"] Jul 06 19:55:46.913216 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:55:46.913192 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc78ef021_0d64_4e88_854b_036f17dd2401.slice/crio-a93aa87c538bf83c75ff4de931235b2bed9d82df62bfd08a5d0ea7268499f3c3 WatchSource:0}: Error finding container a93aa87c538bf83c75ff4de931235b2bed9d82df62bfd08a5d0ea7268499f3c3: Status 404 returned error can't find the container with id a93aa87c538bf83c75ff4de931235b2bed9d82df62bfd08a5d0ea7268499f3c3 Jul 06 19:55:47.096734 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:47.096698 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn" event={"ID":"c78ef021-0d64-4e88-854b-036f17dd2401","Type":"ContainerStarted","Data":"a93aa87c538bf83c75ff4de931235b2bed9d82df62bfd08a5d0ea7268499f3c3"} Jul 06 19:55:52.124808 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:52.124774 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn" event={"ID":"c78ef021-0d64-4e88-854b-036f17dd2401","Type":"ContainerStarted","Data":"9c294fab00ffa4ca323fe0c2eb8c421c1cd3c63a26eed7881f94c1d42a771b5b"} Jul 06 19:55:52.125265 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:52.124895 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn" Jul 06 19:55:52.126563 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:52.126538 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sl8vx" podUID="8e6cf12a-4d1d-433f-93a1-c073560103a9" containerName="manager" containerID="cri-o://b1e79c94bdeb75d69a2df055bd877ccd87ce8e79c3616627a7a66594755f02bb" gracePeriod=2 Jul 06 19:55:52.145749 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:55:52.145711 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-sl8vx\" is forbidden: User \"system:node:ip-10-0-130-172.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-172.ec2.internal' and this object" podUID="8e6cf12a-4d1d-433f-93a1-c073560103a9" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sl8vx" Jul 06 19:55:52.146655 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:52.146615 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn" podStartSLOduration=1.12114992 podStartE2EDuration="6.146600875s" totalImagesPullingTime="5.025450955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:55:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:55:52.143463604 +0000 UTC m=+451.590700323" watchObservedRunningTime="2026-07-06 19:55:52.146600875 +0000 UTC m=+451.593837592" Jul 06 19:55:52.369919 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:52.369896 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sl8vx" Jul 06 19:55:52.372488 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:55:52.372465 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-sl8vx\" is forbidden: User \"system:node:ip-10-0-130-172.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-172.ec2.internal' and this object" podUID="8e6cf12a-4d1d-433f-93a1-c073560103a9" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sl8vx" Jul 06 19:55:52.544457 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:52.544414 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rz6sz\" (UniqueName: \"kubernetes.io/projected/8e6cf12a-4d1d-433f-93a1-c073560103a9-kube-api-access-rz6sz\") pod \"8e6cf12a-4d1d-433f-93a1-c073560103a9\" (UID: \"8e6cf12a-4d1d-433f-93a1-c073560103a9\") " Jul 06 19:55:52.544652 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:52.544544 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e6cf12a-4d1d-433f-93a1-c073560103a9-extensions-socket-volume\") pod \"8e6cf12a-4d1d-433f-93a1-c073560103a9\" (UID: \"8e6cf12a-4d1d-433f-93a1-c073560103a9\") " Jul 06 19:55:52.545087 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:52.545050 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e6cf12a-4d1d-433f-93a1-c073560103a9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8e6cf12a-4d1d-433f-93a1-c073560103a9" (UID: "8e6cf12a-4d1d-433f-93a1-c073560103a9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:55:52.546765 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:52.546736 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e6cf12a-4d1d-433f-93a1-c073560103a9-kube-api-access-rz6sz" (OuterVolumeSpecName: "kube-api-access-rz6sz") pod "8e6cf12a-4d1d-433f-93a1-c073560103a9" (UID: "8e6cf12a-4d1d-433f-93a1-c073560103a9"). InnerVolumeSpecName "kube-api-access-rz6sz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:55:52.645401 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:52.645366 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e6cf12a-4d1d-433f-93a1-c073560103a9-extensions-socket-volume\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:55:52.645401 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:52.645400 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rz6sz\" (UniqueName: \"kubernetes.io/projected/8e6cf12a-4d1d-433f-93a1-c073560103a9-kube-api-access-rz6sz\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:55:53.094906 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:53.094871 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e6cf12a-4d1d-433f-93a1-c073560103a9" path="/var/lib/kubelet/pods/8e6cf12a-4d1d-433f-93a1-c073560103a9/volumes" Jul 06 19:55:53.131952 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:53.131923 2581 generic.go:356] "Generic (PLEG): container finished" podID="8e6cf12a-4d1d-433f-93a1-c073560103a9" containerID="b1e79c94bdeb75d69a2df055bd877ccd87ce8e79c3616627a7a66594755f02bb" exitCode=2 Jul 06 19:55:53.132338 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:53.131971 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sl8vx" Jul 06 19:55:53.132338 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:53.132017 2581 scope.go:117] "RemoveContainer" containerID="b1e79c94bdeb75d69a2df055bd877ccd87ce8e79c3616627a7a66594755f02bb" Jul 06 19:55:53.134228 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:55:53.134203 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-sl8vx\" is forbidden: User \"system:node:ip-10-0-130-172.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-172.ec2.internal' and this object" podUID="8e6cf12a-4d1d-433f-93a1-c073560103a9" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sl8vx" Jul 06 19:55:53.136050 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:55:53.136031 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-sl8vx\" is forbidden: User \"system:node:ip-10-0-130-172.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-172.ec2.internal' and this object" podUID="8e6cf12a-4d1d-433f-93a1-c073560103a9" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-sl8vx" Jul 06 19:55:53.142007 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:53.141986 2581 scope.go:117] "RemoveContainer" containerID="b1e79c94bdeb75d69a2df055bd877ccd87ce8e79c3616627a7a66594755f02bb" Jul 06 19:55:53.142511 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:55:53.142488 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1e79c94bdeb75d69a2df055bd877ccd87ce8e79c3616627a7a66594755f02bb\": container with ID starting with b1e79c94bdeb75d69a2df055bd877ccd87ce8e79c3616627a7a66594755f02bb not found: ID does not exist" containerID="b1e79c94bdeb75d69a2df055bd877ccd87ce8e79c3616627a7a66594755f02bb" Jul 06 19:55:53.142631 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:55:53.142519 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1e79c94bdeb75d69a2df055bd877ccd87ce8e79c3616627a7a66594755f02bb"} err="failed to get container status \"b1e79c94bdeb75d69a2df055bd877ccd87ce8e79c3616627a7a66594755f02bb\": rpc error: code = NotFound desc = could not find container \"b1e79c94bdeb75d69a2df055bd877ccd87ce8e79c3616627a7a66594755f02bb\": container with ID starting with b1e79c94bdeb75d69a2df055bd877ccd87ce8e79c3616627a7a66594755f02bb not found: ID does not exist" Jul 06 19:56:03.134307 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:03.134276 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn" Jul 06 19:56:15.895383 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:15.895347 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn"] Jul 06 19:56:15.895770 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:15.895614 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn" podUID="c78ef021-0d64-4e88-854b-036f17dd2401" containerName="manager" containerID="cri-o://9c294fab00ffa4ca323fe0c2eb8c421c1cd3c63a26eed7881f94c1d42a771b5b" gracePeriod=10 Jul 06 19:56:16.155947 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:16.155889 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn" Jul 06 19:56:16.239812 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:16.239784 2581 generic.go:356] "Generic (PLEG): container finished" podID="c78ef021-0d64-4e88-854b-036f17dd2401" containerID="9c294fab00ffa4ca323fe0c2eb8c421c1cd3c63a26eed7881f94c1d42a771b5b" exitCode=0 Jul 06 19:56:16.239976 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:16.239836 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn" Jul 06 19:56:16.239976 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:16.239871 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn" event={"ID":"c78ef021-0d64-4e88-854b-036f17dd2401","Type":"ContainerDied","Data":"9c294fab00ffa4ca323fe0c2eb8c421c1cd3c63a26eed7881f94c1d42a771b5b"} Jul 06 19:56:16.239976 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:16.239923 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn" event={"ID":"c78ef021-0d64-4e88-854b-036f17dd2401","Type":"ContainerDied","Data":"a93aa87c538bf83c75ff4de931235b2bed9d82df62bfd08a5d0ea7268499f3c3"} Jul 06 19:56:16.239976 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:16.239946 2581 scope.go:117] "RemoveContainer" containerID="9c294fab00ffa4ca323fe0c2eb8c421c1cd3c63a26eed7881f94c1d42a771b5b" Jul 06 19:56:16.247982 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:16.247964 2581 scope.go:117] "RemoveContainer" containerID="9c294fab00ffa4ca323fe0c2eb8c421c1cd3c63a26eed7881f94c1d42a771b5b" Jul 06 19:56:16.248253 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:56:16.248233 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c294fab00ffa4ca323fe0c2eb8c421c1cd3c63a26eed7881f94c1d42a771b5b\": container with ID starting with 9c294fab00ffa4ca323fe0c2eb8c421c1cd3c63a26eed7881f94c1d42a771b5b not found: ID does not exist" containerID="9c294fab00ffa4ca323fe0c2eb8c421c1cd3c63a26eed7881f94c1d42a771b5b" Jul 06 19:56:16.248354 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:16.248265 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c294fab00ffa4ca323fe0c2eb8c421c1cd3c63a26eed7881f94c1d42a771b5b"} err="failed to get container status \"9c294fab00ffa4ca323fe0c2eb8c421c1cd3c63a26eed7881f94c1d42a771b5b\": rpc error: code = NotFound desc = could not find container \"9c294fab00ffa4ca323fe0c2eb8c421c1cd3c63a26eed7881f94c1d42a771b5b\": container with ID starting with 9c294fab00ffa4ca323fe0c2eb8c421c1cd3c63a26eed7881f94c1d42a771b5b not found: ID does not exist" Jul 06 19:56:16.253073 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:16.253055 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnjkv\" (UniqueName: \"kubernetes.io/projected/c78ef021-0d64-4e88-854b-036f17dd2401-kube-api-access-vnjkv\") pod \"c78ef021-0d64-4e88-854b-036f17dd2401\" (UID: \"c78ef021-0d64-4e88-854b-036f17dd2401\") " Jul 06 19:56:16.253175 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:16.253102 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c78ef021-0d64-4e88-854b-036f17dd2401-extensions-socket-volume\") pod \"c78ef021-0d64-4e88-854b-036f17dd2401\" (UID: \"c78ef021-0d64-4e88-854b-036f17dd2401\") " Jul 06 19:56:16.253560 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:16.253538 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c78ef021-0d64-4e88-854b-036f17dd2401-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c78ef021-0d64-4e88-854b-036f17dd2401" (UID: "c78ef021-0d64-4e88-854b-036f17dd2401"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:56:16.254965 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:16.254938 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c78ef021-0d64-4e88-854b-036f17dd2401-kube-api-access-vnjkv" (OuterVolumeSpecName: "kube-api-access-vnjkv") pod "c78ef021-0d64-4e88-854b-036f17dd2401" (UID: "c78ef021-0d64-4e88-854b-036f17dd2401"). InnerVolumeSpecName "kube-api-access-vnjkv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:56:16.354430 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:16.354396 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnjkv\" (UniqueName: \"kubernetes.io/projected/c78ef021-0d64-4e88-854b-036f17dd2401-kube-api-access-vnjkv\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:56:16.354430 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:16.354423 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c78ef021-0d64-4e88-854b-036f17dd2401-extensions-socket-volume\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:56:16.563439 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:16.563405 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn"] Jul 06 19:56:16.567458 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:16.567431 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-87hgn"] Jul 06 19:56:17.094956 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:17.094919 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c78ef021-0d64-4e88-854b-036f17dd2401" path="/var/lib/kubelet/pods/c78ef021-0d64-4e88-854b-036f17dd2401/volumes" Jul 06 19:56:19.535588 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.535552 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh"] Jul 06 19:56:19.536056 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.536039 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c78ef021-0d64-4e88-854b-036f17dd2401" containerName="manager" Jul 06 19:56:19.536101 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.536061 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c78ef021-0d64-4e88-854b-036f17dd2401" containerName="manager" Jul 06 19:56:19.536101 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.536076 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e6cf12a-4d1d-433f-93a1-c073560103a9" containerName="manager" Jul 06 19:56:19.536101 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.536086 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e6cf12a-4d1d-433f-93a1-c073560103a9" containerName="manager" Jul 06 19:56:19.536189 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.536153 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e6cf12a-4d1d-433f-93a1-c073560103a9" containerName="manager" Jul 06 19:56:19.536189 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.536163 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c78ef021-0d64-4e88-854b-036f17dd2401" containerName="manager" Jul 06 19:56:19.540458 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.540440 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.542845 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.542826 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-z8h78\"" Jul 06 19:56:19.559756 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.559732 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh"] Jul 06 19:56:19.582190 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.582163 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/266f87bd-dd56-4772-8d0d-4cfe66e673fb-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.582303 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.582195 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/266f87bd-dd56-4772-8d0d-4cfe66e673fb-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.582303 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.582219 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/266f87bd-dd56-4772-8d0d-4cfe66e673fb-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.582303 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.582242 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/266f87bd-dd56-4772-8d0d-4cfe66e673fb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.582429 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.582311 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hcvh\" (UniqueName: \"kubernetes.io/projected/266f87bd-dd56-4772-8d0d-4cfe66e673fb-kube-api-access-6hcvh\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.582429 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.582366 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/266f87bd-dd56-4772-8d0d-4cfe66e673fb-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.582429 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.582386 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/266f87bd-dd56-4772-8d0d-4cfe66e673fb-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.582551 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.582458 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/266f87bd-dd56-4772-8d0d-4cfe66e673fb-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.582551 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.582479 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/266f87bd-dd56-4772-8d0d-4cfe66e673fb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.683022 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.682985 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/266f87bd-dd56-4772-8d0d-4cfe66e673fb-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.683022 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.683021 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/266f87bd-dd56-4772-8d0d-4cfe66e673fb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.683261 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.683055 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hcvh\" (UniqueName: \"kubernetes.io/projected/266f87bd-dd56-4772-8d0d-4cfe66e673fb-kube-api-access-6hcvh\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.683261 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.683096 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/266f87bd-dd56-4772-8d0d-4cfe66e673fb-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.683261 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.683119 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/266f87bd-dd56-4772-8d0d-4cfe66e673fb-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.683261 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.683201 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/266f87bd-dd56-4772-8d0d-4cfe66e673fb-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.683261 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.683227 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/266f87bd-dd56-4772-8d0d-4cfe66e673fb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.683571 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.683274 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/266f87bd-dd56-4772-8d0d-4cfe66e673fb-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.683571 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.683300 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/266f87bd-dd56-4772-8d0d-4cfe66e673fb-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.683681 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.683590 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/266f87bd-dd56-4772-8d0d-4cfe66e673fb-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.683681 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.683637 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/266f87bd-dd56-4772-8d0d-4cfe66e673fb-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.683808 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.683788 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/266f87bd-dd56-4772-8d0d-4cfe66e673fb-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.683880 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.683860 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/266f87bd-dd56-4772-8d0d-4cfe66e673fb-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.683937 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.683899 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/266f87bd-dd56-4772-8d0d-4cfe66e673fb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.685399 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.685380 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/266f87bd-dd56-4772-8d0d-4cfe66e673fb-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.685657 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.685633 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/266f87bd-dd56-4772-8d0d-4cfe66e673fb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.692068 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.692042 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/266f87bd-dd56-4772-8d0d-4cfe66e673fb-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.692250 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.692228 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hcvh\" (UniqueName: \"kubernetes.io/projected/266f87bd-dd56-4772-8d0d-4cfe66e673fb-kube-api-access-6hcvh\") pod \"maas-default-gateway-openshift-default-56654dcf7-lc6bh\" (UID: \"266f87bd-dd56-4772-8d0d-4cfe66e673fb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.850726 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.850645 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:19.985915 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.985892 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh"] Jul 06 19:56:19.988006 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:56:19.987974 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod266f87bd_dd56_4772_8d0d_4cfe66e673fb.slice/crio-3ff13921994504ec40fe2cfe26ff325a7d3d15bc0a34cc52a90f66fce367e857 WatchSource:0}: Error finding container 3ff13921994504ec40fe2cfe26ff325a7d3d15bc0a34cc52a90f66fce367e857: Status 404 returned error can't find the container with id 3ff13921994504ec40fe2cfe26ff325a7d3d15bc0a34cc52a90f66fce367e857 Jul 06 19:56:19.990014 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.989979 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 19:56:19.990116 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.990060 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 19:56:19.990116 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:19.990106 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 19:56:20.261103 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:20.261074 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" event={"ID":"266f87bd-dd56-4772-8d0d-4cfe66e673fb","Type":"ContainerStarted","Data":"84f8ffc7446b2da28b92eea5c3868b19f42a66161cd3233bd6c92aebcbdc2d2f"} Jul 06 19:56:20.261293 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:20.261121 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" event={"ID":"266f87bd-dd56-4772-8d0d-4cfe66e673fb","Type":"ContainerStarted","Data":"3ff13921994504ec40fe2cfe26ff325a7d3d15bc0a34cc52a90f66fce367e857"} Jul 06 19:56:20.281498 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:20.281449 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" podStartSLOduration=1.281432178 podStartE2EDuration="1.281432178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:56:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:56:20.280469648 +0000 UTC m=+479.727706368" watchObservedRunningTime="2026-07-06 19:56:20.281432178 +0000 UTC m=+479.728668954" Jul 06 19:56:20.851094 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:20.851060 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:21.856451 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:21.856421 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:22.269455 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:22.269424 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:22.270308 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:22.270290 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:56:32.976721 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:32.976687 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qcw5j"] Jul 06 19:56:32.982739 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:32.982722 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qcw5j" Jul 06 19:56:32.985558 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:32.985530 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 19:56:32.986699 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:32.986679 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 19:56:32.986699 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:32.986689 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-pbbgj\"" Jul 06 19:56:32.986853 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:32.986703 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 19:56:32.991835 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:32.991812 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qcw5j"] Jul 06 19:56:33.080056 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:33.080025 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qcw5j"] Jul 06 19:56:33.106046 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:33.106018 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a106a8a1-4eb4-4a08-afff-7f5ddfb84d23-config-file\") pod \"limitador-limitador-64c94fd64f-qcw5j\" (UID: \"a106a8a1-4eb4-4a08-afff-7f5ddfb84d23\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qcw5j" Jul 06 19:56:33.106193 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:33.106114 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9lzk\" (UniqueName: \"kubernetes.io/projected/a106a8a1-4eb4-4a08-afff-7f5ddfb84d23-kube-api-access-l9lzk\") pod \"limitador-limitador-64c94fd64f-qcw5j\" (UID: \"a106a8a1-4eb4-4a08-afff-7f5ddfb84d23\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qcw5j" Jul 06 19:56:33.206601 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:33.206557 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a106a8a1-4eb4-4a08-afff-7f5ddfb84d23-config-file\") pod \"limitador-limitador-64c94fd64f-qcw5j\" (UID: \"a106a8a1-4eb4-4a08-afff-7f5ddfb84d23\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qcw5j" Jul 06 19:56:33.206789 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:33.206672 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9lzk\" (UniqueName: \"kubernetes.io/projected/a106a8a1-4eb4-4a08-afff-7f5ddfb84d23-kube-api-access-l9lzk\") pod \"limitador-limitador-64c94fd64f-qcw5j\" (UID: \"a106a8a1-4eb4-4a08-afff-7f5ddfb84d23\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qcw5j" Jul 06 19:56:33.207155 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:33.207134 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a106a8a1-4eb4-4a08-afff-7f5ddfb84d23-config-file\") pod \"limitador-limitador-64c94fd64f-qcw5j\" (UID: \"a106a8a1-4eb4-4a08-afff-7f5ddfb84d23\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qcw5j" Jul 06 19:56:33.214976 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:33.214954 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9lzk\" (UniqueName: \"kubernetes.io/projected/a106a8a1-4eb4-4a08-afff-7f5ddfb84d23-kube-api-access-l9lzk\") pod \"limitador-limitador-64c94fd64f-qcw5j\" (UID: \"a106a8a1-4eb4-4a08-afff-7f5ddfb84d23\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qcw5j" Jul 06 19:56:33.293272 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:33.293238 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qcw5j" Jul 06 19:56:33.420939 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:33.420906 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qcw5j"] Jul 06 19:56:33.423095 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:56:33.423061 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda106a8a1_4eb4_4a08_afff_7f5ddfb84d23.slice/crio-02d5a9758019ef08ee33ea375670e185c9583bf024e7c19968936b16ad2d4d1f WatchSource:0}: Error finding container 02d5a9758019ef08ee33ea375670e185c9583bf024e7c19968936b16ad2d4d1f: Status 404 returned error can't find the container with id 02d5a9758019ef08ee33ea375670e185c9583bf024e7c19968936b16ad2d4d1f Jul 06 19:56:34.102248 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:34.102189 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-wjk74"] Jul 06 19:56:34.106007 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:34.105985 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wjk74" Jul 06 19:56:34.109302 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:34.109280 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-svfj7\"" Jul 06 19:56:34.114238 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:34.114206 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wjk74"] Jul 06 19:56:34.216780 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:34.216591 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26f56\" (UniqueName: \"kubernetes.io/projected/26735160-84f5-4249-932b-17060955b9d2-kube-api-access-26f56\") pod \"authorino-75d556bd77-wjk74\" (UID: \"26735160-84f5-4249-932b-17060955b9d2\") " pod="kuadrant-system/authorino-75d556bd77-wjk74" Jul 06 19:56:34.318593 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:34.318546 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26f56\" (UniqueName: \"kubernetes.io/projected/26735160-84f5-4249-932b-17060955b9d2-kube-api-access-26f56\") pod \"authorino-75d556bd77-wjk74\" (UID: \"26735160-84f5-4249-932b-17060955b9d2\") " pod="kuadrant-system/authorino-75d556bd77-wjk74" Jul 06 19:56:34.328244 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:34.328208 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qcw5j" event={"ID":"a106a8a1-4eb4-4a08-afff-7f5ddfb84d23","Type":"ContainerStarted","Data":"02d5a9758019ef08ee33ea375670e185c9583bf024e7c19968936b16ad2d4d1f"} Jul 06 19:56:34.328554 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:34.328503 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26f56\" (UniqueName: \"kubernetes.io/projected/26735160-84f5-4249-932b-17060955b9d2-kube-api-access-26f56\") pod \"authorino-75d556bd77-wjk74\" (UID: \"26735160-84f5-4249-932b-17060955b9d2\") " pod="kuadrant-system/authorino-75d556bd77-wjk74" Jul 06 19:56:34.419760 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:34.419672 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wjk74" Jul 06 19:56:34.566575 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:34.566546 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wjk74"] Jul 06 19:56:34.569663 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:56:34.569612 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26735160_84f5_4249_932b_17060955b9d2.slice/crio-df69bd1ade2d0334635e2ea0c1858dabd4b2e0047934b178ce948daf7c134e6e WatchSource:0}: Error finding container df69bd1ade2d0334635e2ea0c1858dabd4b2e0047934b178ce948daf7c134e6e: Status 404 returned error can't find the container with id df69bd1ade2d0334635e2ea0c1858dabd4b2e0047934b178ce948daf7c134e6e Jul 06 19:56:35.334823 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:35.334776 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wjk74" event={"ID":"26735160-84f5-4249-932b-17060955b9d2","Type":"ContainerStarted","Data":"df69bd1ade2d0334635e2ea0c1858dabd4b2e0047934b178ce948daf7c134e6e"} Jul 06 19:56:37.349565 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:37.349528 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qcw5j" event={"ID":"a106a8a1-4eb4-4a08-afff-7f5ddfb84d23","Type":"ContainerStarted","Data":"24465acf63623a104d2434c5134b1730f1b040bb4ddc080bc40defacb604de08"} Jul 06 19:56:37.350041 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:37.349699 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-qcw5j" Jul 06 19:56:37.367091 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:37.367040 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-qcw5j" podStartSLOduration=2.439542118 podStartE2EDuration="5.367020262s" totalImagesPullingTime="2.927478144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:56:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:56:37.366158708 +0000 UTC m=+496.813395426" watchObservedRunningTime="2026-07-06 19:56:37.367020262 +0000 UTC m=+496.814256981" Jul 06 19:56:38.355595 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:38.355497 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wjk74" event={"ID":"26735160-84f5-4249-932b-17060955b9d2","Type":"ContainerStarted","Data":"b8b85342d5aa3ab23a0b556880a09307208d4703c2c3234aa0fb298c04b0b257"} Jul 06 19:56:38.372066 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:38.372014 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-wjk74" podStartSLOduration=0.84773173 podStartE2EDuration="4.371998799s" totalImagesPullingTime="3.524267069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:56:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:56:38.370655322 +0000 UTC m=+497.817892041" watchObservedRunningTime="2026-07-06 19:56:38.371998799 +0000 UTC m=+497.819235518" Jul 06 19:56:48.356560 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:48.356527 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-qcw5j" Jul 06 19:56:50.059443 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:50.059409 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qcw5j"] Jul 06 19:56:50.059918 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:50.059671 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-qcw5j" podUID="a106a8a1-4eb4-4a08-afff-7f5ddfb84d23" containerName="limitador" containerID="cri-o://24465acf63623a104d2434c5134b1730f1b040bb4ddc080bc40defacb604de08" gracePeriod=30 Jul 06 19:56:50.601622 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:50.601596 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qcw5j" Jul 06 19:56:50.664800 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:50.664726 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9lzk\" (UniqueName: \"kubernetes.io/projected/a106a8a1-4eb4-4a08-afff-7f5ddfb84d23-kube-api-access-l9lzk\") pod \"a106a8a1-4eb4-4a08-afff-7f5ddfb84d23\" (UID: \"a106a8a1-4eb4-4a08-afff-7f5ddfb84d23\") " Jul 06 19:56:50.664800 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:50.664785 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a106a8a1-4eb4-4a08-afff-7f5ddfb84d23-config-file\") pod \"a106a8a1-4eb4-4a08-afff-7f5ddfb84d23\" (UID: \"a106a8a1-4eb4-4a08-afff-7f5ddfb84d23\") " Jul 06 19:56:50.665142 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:50.665121 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a106a8a1-4eb4-4a08-afff-7f5ddfb84d23-config-file" (OuterVolumeSpecName: "config-file") pod "a106a8a1-4eb4-4a08-afff-7f5ddfb84d23" (UID: "a106a8a1-4eb4-4a08-afff-7f5ddfb84d23"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:56:50.666773 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:50.666754 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a106a8a1-4eb4-4a08-afff-7f5ddfb84d23-kube-api-access-l9lzk" (OuterVolumeSpecName: "kube-api-access-l9lzk") pod "a106a8a1-4eb4-4a08-afff-7f5ddfb84d23" (UID: "a106a8a1-4eb4-4a08-afff-7f5ddfb84d23"). InnerVolumeSpecName "kube-api-access-l9lzk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:56:50.766161 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:50.766115 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9lzk\" (UniqueName: \"kubernetes.io/projected/a106a8a1-4eb4-4a08-afff-7f5ddfb84d23-kube-api-access-l9lzk\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:56:50.766161 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:50.766153 2581 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a106a8a1-4eb4-4a08-afff-7f5ddfb84d23-config-file\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:56:51.413934 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:51.413845 2581 generic.go:356] "Generic (PLEG): container finished" podID="a106a8a1-4eb4-4a08-afff-7f5ddfb84d23" containerID="24465acf63623a104d2434c5134b1730f1b040bb4ddc080bc40defacb604de08" exitCode=0 Jul 06 19:56:51.413934 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:51.413884 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qcw5j" event={"ID":"a106a8a1-4eb4-4a08-afff-7f5ddfb84d23","Type":"ContainerDied","Data":"24465acf63623a104d2434c5134b1730f1b040bb4ddc080bc40defacb604de08"} Jul 06 19:56:51.413934 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:51.413897 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qcw5j" Jul 06 19:56:51.414503 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:51.413940 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qcw5j" event={"ID":"a106a8a1-4eb4-4a08-afff-7f5ddfb84d23","Type":"ContainerDied","Data":"02d5a9758019ef08ee33ea375670e185c9583bf024e7c19968936b16ad2d4d1f"} Jul 06 19:56:51.414503 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:51.413970 2581 scope.go:117] "RemoveContainer" containerID="24465acf63623a104d2434c5134b1730f1b040bb4ddc080bc40defacb604de08" Jul 06 19:56:51.422197 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:51.422182 2581 scope.go:117] "RemoveContainer" containerID="24465acf63623a104d2434c5134b1730f1b040bb4ddc080bc40defacb604de08" Jul 06 19:56:51.422456 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:56:51.422439 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24465acf63623a104d2434c5134b1730f1b040bb4ddc080bc40defacb604de08\": container with ID starting with 24465acf63623a104d2434c5134b1730f1b040bb4ddc080bc40defacb604de08 not found: ID does not exist" containerID="24465acf63623a104d2434c5134b1730f1b040bb4ddc080bc40defacb604de08" Jul 06 19:56:51.422519 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:51.422463 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24465acf63623a104d2434c5134b1730f1b040bb4ddc080bc40defacb604de08"} err="failed to get container status \"24465acf63623a104d2434c5134b1730f1b040bb4ddc080bc40defacb604de08\": rpc error: code = NotFound desc = could not find container \"24465acf63623a104d2434c5134b1730f1b040bb4ddc080bc40defacb604de08\": container with ID starting with 24465acf63623a104d2434c5134b1730f1b040bb4ddc080bc40defacb604de08 not found: ID does not exist" Jul 06 19:56:51.430305 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:51.430267 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qcw5j"] Jul 06 19:56:51.433272 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:51.433252 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qcw5j"] Jul 06 19:56:53.095012 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:53.094979 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a106a8a1-4eb4-4a08-afff-7f5ddfb84d23" path="/var/lib/kubelet/pods/a106a8a1-4eb4-4a08-afff-7f5ddfb84d23/volumes" Jul 06 19:56:53.156959 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:53.156926 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-vzglc"] Jul 06 19:56:53.157947 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:53.157916 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a106a8a1-4eb4-4a08-afff-7f5ddfb84d23" containerName="limitador" Jul 06 19:56:53.157947 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:53.157949 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a106a8a1-4eb4-4a08-afff-7f5ddfb84d23" containerName="limitador" Jul 06 19:56:53.162052 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:53.158183 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a106a8a1-4eb4-4a08-afff-7f5ddfb84d23" containerName="limitador" Jul 06 19:56:53.166897 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:53.166872 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-vzglc" Jul 06 19:56:53.170201 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:53.170164 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 06 19:56:53.170461 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:53.170394 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-kc5n2\"" Jul 06 19:56:53.172694 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:53.172672 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-vzglc"] Jul 06 19:56:53.187283 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:53.187258 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2z6t\" (UniqueName: \"kubernetes.io/projected/91550056-791c-447d-8bf2-c83d2b11e539-kube-api-access-b2z6t\") pod \"postgres-69fb4594cf-vzglc\" (UID: \"91550056-791c-447d-8bf2-c83d2b11e539\") " pod="opendatahub/postgres-69fb4594cf-vzglc" Jul 06 19:56:53.187419 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:53.187386 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/91550056-791c-447d-8bf2-c83d2b11e539-data\") pod \"postgres-69fb4594cf-vzglc\" (UID: \"91550056-791c-447d-8bf2-c83d2b11e539\") " pod="opendatahub/postgres-69fb4594cf-vzglc" Jul 06 19:56:53.288676 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:53.288635 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2z6t\" (UniqueName: \"kubernetes.io/projected/91550056-791c-447d-8bf2-c83d2b11e539-kube-api-access-b2z6t\") pod \"postgres-69fb4594cf-vzglc\" (UID: \"91550056-791c-447d-8bf2-c83d2b11e539\") " pod="opendatahub/postgres-69fb4594cf-vzglc" Jul 06 19:56:53.288868 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:53.288766 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/91550056-791c-447d-8bf2-c83d2b11e539-data\") pod \"postgres-69fb4594cf-vzglc\" (UID: \"91550056-791c-447d-8bf2-c83d2b11e539\") " pod="opendatahub/postgres-69fb4594cf-vzglc" Jul 06 19:56:53.289129 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:53.289108 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/91550056-791c-447d-8bf2-c83d2b11e539-data\") pod \"postgres-69fb4594cf-vzglc\" (UID: \"91550056-791c-447d-8bf2-c83d2b11e539\") " pod="opendatahub/postgres-69fb4594cf-vzglc" Jul 06 19:56:53.296377 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:53.296352 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2z6t\" (UniqueName: \"kubernetes.io/projected/91550056-791c-447d-8bf2-c83d2b11e539-kube-api-access-b2z6t\") pod \"postgres-69fb4594cf-vzglc\" (UID: \"91550056-791c-447d-8bf2-c83d2b11e539\") " pod="opendatahub/postgres-69fb4594cf-vzglc" Jul 06 19:56:53.478980 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:53.478902 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-vzglc" Jul 06 19:56:53.609085 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:53.609005 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-vzglc"] Jul 06 19:56:53.611836 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:56:53.611810 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91550056_791c_447d_8bf2_c83d2b11e539.slice/crio-abce2b871d44fa8ef08f09eb1614943f0ac2c2de27beff787b4235726481bd4d WatchSource:0}: Error finding container abce2b871d44fa8ef08f09eb1614943f0ac2c2de27beff787b4235726481bd4d: Status 404 returned error can't find the container with id abce2b871d44fa8ef08f09eb1614943f0ac2c2de27beff787b4235726481bd4d Jul 06 19:56:54.432287 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:54.432247 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-vzglc" event={"ID":"91550056-791c-447d-8bf2-c83d2b11e539","Type":"ContainerStarted","Data":"abce2b871d44fa8ef08f09eb1614943f0ac2c2de27beff787b4235726481bd4d"} Jul 06 19:56:58.456503 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:58.456467 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-vzglc" event={"ID":"91550056-791c-447d-8bf2-c83d2b11e539","Type":"ContainerStarted","Data":"261067c22135e3337308969e869cd5c5ca0267cc6604127a5515f1cb4801b43e"} Jul 06 19:56:58.456939 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:58.456536 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-vzglc" Jul 06 19:56:58.472066 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:56:58.471881 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-vzglc" podStartSLOduration=0.741256963 podStartE2EDuration="5.471861995s" totalImagesPullingTime="4.730605032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:56:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:56:58.471054507 +0000 UTC m=+517.918291225" watchObservedRunningTime="2026-07-06 19:56:58.471861995 +0000 UTC m=+517.919098713" Jul 06 19:57:04.492474 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:04.492443 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-vzglc" Jul 06 19:57:05.416121 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:05.416086 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-779449988c-44d5k"] Jul 06 19:57:05.421867 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:05.421846 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-779449988c-44d5k" Jul 06 19:57:05.424443 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:05.424405 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 06 19:57:05.426543 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:05.426524 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-779449988c-44d5k"] Jul 06 19:57:05.496870 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:05.496839 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bz5x\" (UniqueName: \"kubernetes.io/projected/33bae915-3dbf-4a90-8d7b-f7980a8b7e59-kube-api-access-7bz5x\") pod \"authorino-779449988c-44d5k\" (UID: \"33bae915-3dbf-4a90-8d7b-f7980a8b7e59\") " pod="kuadrant-system/authorino-779449988c-44d5k" Jul 06 19:57:05.497211 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:05.496904 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/33bae915-3dbf-4a90-8d7b-f7980a8b7e59-tls-cert\") pod \"authorino-779449988c-44d5k\" (UID: \"33bae915-3dbf-4a90-8d7b-f7980a8b7e59\") " pod="kuadrant-system/authorino-779449988c-44d5k" Jul 06 19:57:05.597893 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:05.597857 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/33bae915-3dbf-4a90-8d7b-f7980a8b7e59-tls-cert\") pod \"authorino-779449988c-44d5k\" (UID: \"33bae915-3dbf-4a90-8d7b-f7980a8b7e59\") " pod="kuadrant-system/authorino-779449988c-44d5k" Jul 06 19:57:05.598039 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:05.597953 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bz5x\" (UniqueName: \"kubernetes.io/projected/33bae915-3dbf-4a90-8d7b-f7980a8b7e59-kube-api-access-7bz5x\") pod \"authorino-779449988c-44d5k\" (UID: \"33bae915-3dbf-4a90-8d7b-f7980a8b7e59\") " pod="kuadrant-system/authorino-779449988c-44d5k" Jul 06 19:57:05.600258 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:05.600230 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/33bae915-3dbf-4a90-8d7b-f7980a8b7e59-tls-cert\") pod \"authorino-779449988c-44d5k\" (UID: \"33bae915-3dbf-4a90-8d7b-f7980a8b7e59\") " pod="kuadrant-system/authorino-779449988c-44d5k" Jul 06 19:57:05.606569 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:05.606545 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bz5x\" (UniqueName: \"kubernetes.io/projected/33bae915-3dbf-4a90-8d7b-f7980a8b7e59-kube-api-access-7bz5x\") pod \"authorino-779449988c-44d5k\" (UID: \"33bae915-3dbf-4a90-8d7b-f7980a8b7e59\") " pod="kuadrant-system/authorino-779449988c-44d5k" Jul 06 19:57:05.731476 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:05.731398 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-779449988c-44d5k" Jul 06 19:57:05.867241 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:05.864599 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-779449988c-44d5k"] Jul 06 19:57:06.494713 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:06.494618 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-779449988c-44d5k" event={"ID":"33bae915-3dbf-4a90-8d7b-f7980a8b7e59","Type":"ContainerStarted","Data":"83f0263c1c7da03774d39b91dba3d2ed38e3d3967dcaa3482c7b9cc82cbc8b24"} Jul 06 19:57:06.494713 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:06.494662 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-779449988c-44d5k" event={"ID":"33bae915-3dbf-4a90-8d7b-f7980a8b7e59","Type":"ContainerStarted","Data":"663b3ea8c9bc3af70f96aa95f6504827f041ad2f5e909b1a61804aaf3873425d"} Jul 06 19:57:06.511608 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:06.511558 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-779449988c-44d5k" podStartSLOduration=1.153347602 podStartE2EDuration="1.511543307s" totalImagesPullingTime="358.195705ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:57:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:57:06.510308963 +0000 UTC m=+525.957545684" watchObservedRunningTime="2026-07-06 19:57:06.511543307 +0000 UTC m=+525.958780025" Jul 06 19:57:06.538259 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:06.538213 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wjk74"] Jul 06 19:57:06.538508 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:06.538463 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-wjk74" podUID="26735160-84f5-4249-932b-17060955b9d2" containerName="authorino" containerID="cri-o://b8b85342d5aa3ab23a0b556880a09307208d4703c2c3234aa0fb298c04b0b257" gracePeriod=30 Jul 06 19:57:06.803145 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:06.803119 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wjk74" Jul 06 19:57:06.910552 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:06.910511 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26f56\" (UniqueName: \"kubernetes.io/projected/26735160-84f5-4249-932b-17060955b9d2-kube-api-access-26f56\") pod \"26735160-84f5-4249-932b-17060955b9d2\" (UID: \"26735160-84f5-4249-932b-17060955b9d2\") " Jul 06 19:57:06.912516 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:06.912493 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26735160-84f5-4249-932b-17060955b9d2-kube-api-access-26f56" (OuterVolumeSpecName: "kube-api-access-26f56") pod "26735160-84f5-4249-932b-17060955b9d2" (UID: "26735160-84f5-4249-932b-17060955b9d2"). InnerVolumeSpecName "kube-api-access-26f56". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:57:07.011446 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:07.011409 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26f56\" (UniqueName: \"kubernetes.io/projected/26735160-84f5-4249-932b-17060955b9d2-kube-api-access-26f56\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:57:07.500669 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:07.500568 2581 generic.go:356] "Generic (PLEG): container finished" podID="26735160-84f5-4249-932b-17060955b9d2" containerID="b8b85342d5aa3ab23a0b556880a09307208d4703c2c3234aa0fb298c04b0b257" exitCode=0 Jul 06 19:57:07.500669 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:07.500607 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wjk74" Jul 06 19:57:07.500669 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:07.500660 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wjk74" event={"ID":"26735160-84f5-4249-932b-17060955b9d2","Type":"ContainerDied","Data":"b8b85342d5aa3ab23a0b556880a09307208d4703c2c3234aa0fb298c04b0b257"} Jul 06 19:57:07.500954 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:07.500700 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wjk74" event={"ID":"26735160-84f5-4249-932b-17060955b9d2","Type":"ContainerDied","Data":"df69bd1ade2d0334635e2ea0c1858dabd4b2e0047934b178ce948daf7c134e6e"} Jul 06 19:57:07.500954 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:07.500717 2581 scope.go:117] "RemoveContainer" containerID="b8b85342d5aa3ab23a0b556880a09307208d4703c2c3234aa0fb298c04b0b257" Jul 06 19:57:07.509159 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:07.509139 2581 scope.go:117] "RemoveContainer" containerID="b8b85342d5aa3ab23a0b556880a09307208d4703c2c3234aa0fb298c04b0b257" Jul 06 19:57:07.509462 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:57:07.509443 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8b85342d5aa3ab23a0b556880a09307208d4703c2c3234aa0fb298c04b0b257\": container with ID starting with b8b85342d5aa3ab23a0b556880a09307208d4703c2c3234aa0fb298c04b0b257 not found: ID does not exist" containerID="b8b85342d5aa3ab23a0b556880a09307208d4703c2c3234aa0fb298c04b0b257" Jul 06 19:57:07.509539 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:07.509474 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8b85342d5aa3ab23a0b556880a09307208d4703c2c3234aa0fb298c04b0b257"} err="failed to get container status \"b8b85342d5aa3ab23a0b556880a09307208d4703c2c3234aa0fb298c04b0b257\": rpc error: code = NotFound desc = could not find container \"b8b85342d5aa3ab23a0b556880a09307208d4703c2c3234aa0fb298c04b0b257\": container with ID starting with b8b85342d5aa3ab23a0b556880a09307208d4703c2c3234aa0fb298c04b0b257 not found: ID does not exist" Jul 06 19:57:07.515969 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:07.515944 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wjk74"] Jul 06 19:57:07.519851 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:07.519831 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wjk74"] Jul 06 19:57:09.095215 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:09.095183 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26735160-84f5-4249-932b-17060955b9d2" path="/var/lib/kubelet/pods/26735160-84f5-4249-932b-17060955b9d2/volumes" Jul 06 19:57:10.873445 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:10.873409 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d8f99bf58-t2pz4"] Jul 06 19:57:10.873813 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:10.873775 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26735160-84f5-4249-932b-17060955b9d2" containerName="authorino" Jul 06 19:57:10.873813 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:10.873786 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="26735160-84f5-4249-932b-17060955b9d2" containerName="authorino" Jul 06 19:57:10.873891 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:10.873859 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="26735160-84f5-4249-932b-17060955b9d2" containerName="authorino" Jul 06 19:57:10.931137 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:10.931100 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d8f99bf58-t2pz4"] Jul 06 19:57:10.931295 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:10.931212 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d8f99bf58-t2pz4" Jul 06 19:57:10.933993 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:10.933963 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 19:57:10.934136 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:10.934012 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 19:57:10.934267 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:10.934254 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-sx4sv\"" Jul 06 19:57:11.043940 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:11.043909 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7588cc9ccd-7m9ws"] Jul 06 19:57:11.049101 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:11.049076 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n5zn\" (UniqueName: \"kubernetes.io/projected/860eb73c-7f47-4106-be99-3d4f2b2d1f2b-kube-api-access-7n5zn\") pod \"maas-controller-d8f99bf58-t2pz4\" (UID: \"860eb73c-7f47-4106-be99-3d4f2b2d1f2b\") " pod="opendatahub/maas-controller-d8f99bf58-t2pz4" Jul 06 19:57:11.049230 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:11.049198 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/860eb73c-7f47-4106-be99-3d4f2b2d1f2b-webhook-cert\") pod \"maas-controller-d8f99bf58-t2pz4\" (UID: \"860eb73c-7f47-4106-be99-3d4f2b2d1f2b\") " pod="opendatahub/maas-controller-d8f99bf58-t2pz4" Jul 06 19:57:11.065382 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:11.065350 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7588cc9ccd-7m9ws"] Jul 06 19:57:11.065548 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:11.065462 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7588cc9ccd-7m9ws" Jul 06 19:57:11.149827 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:11.149750 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6qc5\" (UniqueName: \"kubernetes.io/projected/c2dbcb90-1a1e-4d77-89f6-e408cbba8728-kube-api-access-w6qc5\") pod \"maas-controller-7588cc9ccd-7m9ws\" (UID: \"c2dbcb90-1a1e-4d77-89f6-e408cbba8728\") " pod="opendatahub/maas-controller-7588cc9ccd-7m9ws" Jul 06 19:57:11.149827 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:11.149801 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2dbcb90-1a1e-4d77-89f6-e408cbba8728-webhook-cert\") pod \"maas-controller-7588cc9ccd-7m9ws\" (UID: \"c2dbcb90-1a1e-4d77-89f6-e408cbba8728\") " pod="opendatahub/maas-controller-7588cc9ccd-7m9ws" Jul 06 19:57:11.150021 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:11.149865 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/860eb73c-7f47-4106-be99-3d4f2b2d1f2b-webhook-cert\") pod \"maas-controller-d8f99bf58-t2pz4\" (UID: \"860eb73c-7f47-4106-be99-3d4f2b2d1f2b\") " pod="opendatahub/maas-controller-d8f99bf58-t2pz4" Jul 06 19:57:11.150021 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:11.149924 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n5zn\" (UniqueName: \"kubernetes.io/projected/860eb73c-7f47-4106-be99-3d4f2b2d1f2b-kube-api-access-7n5zn\") pod \"maas-controller-d8f99bf58-t2pz4\" (UID: \"860eb73c-7f47-4106-be99-3d4f2b2d1f2b\") " pod="opendatahub/maas-controller-d8f99bf58-t2pz4" Jul 06 19:57:11.152333 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:11.152305 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/860eb73c-7f47-4106-be99-3d4f2b2d1f2b-webhook-cert\") pod \"maas-controller-d8f99bf58-t2pz4\" (UID: \"860eb73c-7f47-4106-be99-3d4f2b2d1f2b\") " pod="opendatahub/maas-controller-d8f99bf58-t2pz4" Jul 06 19:57:11.159549 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:11.159524 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n5zn\" (UniqueName: \"kubernetes.io/projected/860eb73c-7f47-4106-be99-3d4f2b2d1f2b-kube-api-access-7n5zn\") pod \"maas-controller-d8f99bf58-t2pz4\" (UID: \"860eb73c-7f47-4106-be99-3d4f2b2d1f2b\") " pod="opendatahub/maas-controller-d8f99bf58-t2pz4" Jul 06 19:57:11.243705 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:11.243676 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d8f99bf58-t2pz4" Jul 06 19:57:11.251139 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:11.251112 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6qc5\" (UniqueName: \"kubernetes.io/projected/c2dbcb90-1a1e-4d77-89f6-e408cbba8728-kube-api-access-w6qc5\") pod \"maas-controller-7588cc9ccd-7m9ws\" (UID: \"c2dbcb90-1a1e-4d77-89f6-e408cbba8728\") " pod="opendatahub/maas-controller-7588cc9ccd-7m9ws" Jul 06 19:57:11.251249 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:11.251163 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2dbcb90-1a1e-4d77-89f6-e408cbba8728-webhook-cert\") pod \"maas-controller-7588cc9ccd-7m9ws\" (UID: \"c2dbcb90-1a1e-4d77-89f6-e408cbba8728\") " pod="opendatahub/maas-controller-7588cc9ccd-7m9ws" Jul 06 19:57:11.253540 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:11.253516 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2dbcb90-1a1e-4d77-89f6-e408cbba8728-webhook-cert\") pod \"maas-controller-7588cc9ccd-7m9ws\" (UID: \"c2dbcb90-1a1e-4d77-89f6-e408cbba8728\") " pod="opendatahub/maas-controller-7588cc9ccd-7m9ws" Jul 06 19:57:11.259913 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:11.259880 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6qc5\" (UniqueName: \"kubernetes.io/projected/c2dbcb90-1a1e-4d77-89f6-e408cbba8728-kube-api-access-w6qc5\") pod \"maas-controller-7588cc9ccd-7m9ws\" (UID: \"c2dbcb90-1a1e-4d77-89f6-e408cbba8728\") " pod="opendatahub/maas-controller-7588cc9ccd-7m9ws" Jul 06 19:57:11.376981 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:11.376945 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7588cc9ccd-7m9ws" Jul 06 19:57:11.406347 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:11.406297 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d8f99bf58-t2pz4"] Jul 06 19:57:11.409477 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:57:11.409453 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod860eb73c_7f47_4106_be99_3d4f2b2d1f2b.slice/crio-2cc8bfd4a9eb67c475906f04fe5322721523436befb9d1db24630a24a69e41a2 WatchSource:0}: Error finding container 2cc8bfd4a9eb67c475906f04fe5322721523436befb9d1db24630a24a69e41a2: Status 404 returned error can't find the container with id 2cc8bfd4a9eb67c475906f04fe5322721523436befb9d1db24630a24a69e41a2 Jul 06 19:57:11.521758 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:11.521725 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d8f99bf58-t2pz4" event={"ID":"860eb73c-7f47-4106-be99-3d4f2b2d1f2b","Type":"ContainerStarted","Data":"2cc8bfd4a9eb67c475906f04fe5322721523436befb9d1db24630a24a69e41a2"} Jul 06 19:57:11.534881 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:11.534859 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7588cc9ccd-7m9ws"] Jul 06 19:57:11.536818 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:57:11.536796 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2dbcb90_1a1e_4d77_89f6_e408cbba8728.slice/crio-83c022cfbad4f7487c6e9e0e9938e2ae36cdd014ec8f88f6181061b89c598c0d WatchSource:0}: Error finding container 83c022cfbad4f7487c6e9e0e9938e2ae36cdd014ec8f88f6181061b89c598c0d: Status 404 returned error can't find the container with id 83c022cfbad4f7487c6e9e0e9938e2ae36cdd014ec8f88f6181061b89c598c0d Jul 06 19:57:12.530810 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:12.530772 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7588cc9ccd-7m9ws" event={"ID":"c2dbcb90-1a1e-4d77-89f6-e408cbba8728","Type":"ContainerStarted","Data":"83c022cfbad4f7487c6e9e0e9938e2ae36cdd014ec8f88f6181061b89c598c0d"} Jul 06 19:57:15.549623 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:15.549584 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7588cc9ccd-7m9ws" event={"ID":"c2dbcb90-1a1e-4d77-89f6-e408cbba8728","Type":"ContainerStarted","Data":"6469161b3408f6311e3e475a599e9813687634849322da8ff5e632921bba612a"} Jul 06 19:57:15.550189 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:15.549748 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7588cc9ccd-7m9ws" Jul 06 19:57:15.551145 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:15.551116 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d8f99bf58-t2pz4" event={"ID":"860eb73c-7f47-4106-be99-3d4f2b2d1f2b","Type":"ContainerStarted","Data":"31dbe565623a808e3765feea0aea551443975da41e0083e1641e96ffeb9aead9"} Jul 06 19:57:15.551230 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:15.551180 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d8f99bf58-t2pz4" Jul 06 19:57:15.568731 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:15.568683 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7588cc9ccd-7m9ws" podStartSLOduration=1.101810208 podStartE2EDuration="4.568667486s" totalImagesPullingTime="3.466857278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:57:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:57:15.567237901 +0000 UTC m=+535.014474621" watchObservedRunningTime="2026-07-06 19:57:15.568667486 +0000 UTC m=+535.015904182" Jul 06 19:57:15.586480 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:15.586429 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d8f99bf58-t2pz4" podStartSLOduration=1.982262829 podStartE2EDuration="5.586413614s" totalImagesPullingTime="3.604150785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:57:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:57:15.585254334 +0000 UTC m=+535.032491053" watchObservedRunningTime="2026-07-06 19:57:15.586413614 +0000 UTC m=+535.033650332" Jul 06 19:57:26.562106 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:26.562068 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d8f99bf58-t2pz4" Jul 06 19:57:26.562566 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:26.562374 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7588cc9ccd-7m9ws" Jul 06 19:57:26.619303 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:26.619270 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d8f99bf58-t2pz4"] Jul 06 19:57:26.619545 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:26.619520 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-d8f99bf58-t2pz4" podUID="860eb73c-7f47-4106-be99-3d4f2b2d1f2b" containerName="manager" containerID="cri-o://31dbe565623a808e3765feea0aea551443975da41e0083e1641e96ffeb9aead9" gracePeriod=30 Jul 06 19:57:26.876719 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:26.876693 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d8f99bf58-t2pz4" Jul 06 19:57:27.000295 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:27.000254 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7n5zn\" (UniqueName: \"kubernetes.io/projected/860eb73c-7f47-4106-be99-3d4f2b2d1f2b-kube-api-access-7n5zn\") pod \"860eb73c-7f47-4106-be99-3d4f2b2d1f2b\" (UID: \"860eb73c-7f47-4106-be99-3d4f2b2d1f2b\") " Jul 06 19:57:27.000537 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:27.000430 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/860eb73c-7f47-4106-be99-3d4f2b2d1f2b-webhook-cert\") pod \"860eb73c-7f47-4106-be99-3d4f2b2d1f2b\" (UID: \"860eb73c-7f47-4106-be99-3d4f2b2d1f2b\") " Jul 06 19:57:27.002584 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:27.002548 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/860eb73c-7f47-4106-be99-3d4f2b2d1f2b-kube-api-access-7n5zn" (OuterVolumeSpecName: "kube-api-access-7n5zn") pod "860eb73c-7f47-4106-be99-3d4f2b2d1f2b" (UID: "860eb73c-7f47-4106-be99-3d4f2b2d1f2b"). InnerVolumeSpecName "kube-api-access-7n5zn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:57:27.002708 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:27.002595 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/860eb73c-7f47-4106-be99-3d4f2b2d1f2b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "860eb73c-7f47-4106-be99-3d4f2b2d1f2b" (UID: "860eb73c-7f47-4106-be99-3d4f2b2d1f2b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:57:27.101389 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:27.101358 2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/860eb73c-7f47-4106-be99-3d4f2b2d1f2b-webhook-cert\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:57:27.101389 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:27.101390 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7n5zn\" (UniqueName: \"kubernetes.io/projected/860eb73c-7f47-4106-be99-3d4f2b2d1f2b-kube-api-access-7n5zn\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 19:57:27.614656 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:27.614623 2581 generic.go:356] "Generic (PLEG): container finished" podID="860eb73c-7f47-4106-be99-3d4f2b2d1f2b" containerID="31dbe565623a808e3765feea0aea551443975da41e0083e1641e96ffeb9aead9" exitCode=0 Jul 06 19:57:27.615083 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:27.614676 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d8f99bf58-t2pz4" event={"ID":"860eb73c-7f47-4106-be99-3d4f2b2d1f2b","Type":"ContainerDied","Data":"31dbe565623a808e3765feea0aea551443975da41e0083e1641e96ffeb9aead9"} Jul 06 19:57:27.615083 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:27.614678 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d8f99bf58-t2pz4" Jul 06 19:57:27.615083 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:27.614717 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d8f99bf58-t2pz4" event={"ID":"860eb73c-7f47-4106-be99-3d4f2b2d1f2b","Type":"ContainerDied","Data":"2cc8bfd4a9eb67c475906f04fe5322721523436befb9d1db24630a24a69e41a2"} Jul 06 19:57:27.615083 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:27.614738 2581 scope.go:117] "RemoveContainer" containerID="31dbe565623a808e3765feea0aea551443975da41e0083e1641e96ffeb9aead9" Jul 06 19:57:27.623210 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:27.623015 2581 scope.go:117] "RemoveContainer" containerID="31dbe565623a808e3765feea0aea551443975da41e0083e1641e96ffeb9aead9" Jul 06 19:57:27.623355 ip-10-0-130-172 kubenswrapper[2581]: E0706 19:57:27.623310 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31dbe565623a808e3765feea0aea551443975da41e0083e1641e96ffeb9aead9\": container with ID starting with 31dbe565623a808e3765feea0aea551443975da41e0083e1641e96ffeb9aead9 not found: ID does not exist" containerID="31dbe565623a808e3765feea0aea551443975da41e0083e1641e96ffeb9aead9" Jul 06 19:57:27.623403 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:27.623364 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31dbe565623a808e3765feea0aea551443975da41e0083e1641e96ffeb9aead9"} err="failed to get container status \"31dbe565623a808e3765feea0aea551443975da41e0083e1641e96ffeb9aead9\": rpc error: code = NotFound desc = could not find container \"31dbe565623a808e3765feea0aea551443975da41e0083e1641e96ffeb9aead9\": container with ID starting with 31dbe565623a808e3765feea0aea551443975da41e0083e1641e96ffeb9aead9 not found: ID does not exist" Jul 06 19:57:27.633672 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:27.633637 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d8f99bf58-t2pz4"] Jul 06 19:57:27.641977 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:27.641947 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-d8f99bf58-t2pz4"] Jul 06 19:57:29.095085 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:29.095053 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="860eb73c-7f47-4106-be99-3d4f2b2d1f2b" path="/var/lib/kubelet/pods/860eb73c-7f47-4106-be99-3d4f2b2d1f2b/volumes" Jul 06 19:57:48.219483 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.219447 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-5846b87474-2rgtl"] Jul 06 19:57:48.219933 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.219842 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="860eb73c-7f47-4106-be99-3d4f2b2d1f2b" containerName="manager" Jul 06 19:57:48.219933 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.219853 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="860eb73c-7f47-4106-be99-3d4f2b2d1f2b" containerName="manager" Jul 06 19:57:48.219933 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.219928 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="860eb73c-7f47-4106-be99-3d4f2b2d1f2b" containerName="manager" Jul 06 19:57:48.224890 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.224869 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5846b87474-2rgtl" Jul 06 19:57:48.228006 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.227985 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 06 19:57:48.237128 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.237104 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5846b87474-2rgtl"] Jul 06 19:57:48.283005 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.282971 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28982306-24aa-4afa-926e-05866cc42c51-openshift-service-ca\") pod \"maas-api-5846b87474-2rgtl\" (UID: \"28982306-24aa-4afa-926e-05866cc42c51\") " pod="opendatahub/maas-api-5846b87474-2rgtl" Jul 06 19:57:48.283197 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.283031 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28982306-24aa-4afa-926e-05866cc42c51-maas-api-tls\") pod \"maas-api-5846b87474-2rgtl\" (UID: \"28982306-24aa-4afa-926e-05866cc42c51\") " pod="opendatahub/maas-api-5846b87474-2rgtl" Jul 06 19:57:48.283197 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.283080 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnmc4\" (UniqueName: \"kubernetes.io/projected/28982306-24aa-4afa-926e-05866cc42c51-kube-api-access-tnmc4\") pod \"maas-api-5846b87474-2rgtl\" (UID: \"28982306-24aa-4afa-926e-05866cc42c51\") " pod="opendatahub/maas-api-5846b87474-2rgtl" Jul 06 19:57:48.384591 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.384556 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28982306-24aa-4afa-926e-05866cc42c51-maas-api-tls\") pod \"maas-api-5846b87474-2rgtl\" (UID: \"28982306-24aa-4afa-926e-05866cc42c51\") " pod="opendatahub/maas-api-5846b87474-2rgtl" Jul 06 19:57:48.384785 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.384610 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnmc4\" (UniqueName: \"kubernetes.io/projected/28982306-24aa-4afa-926e-05866cc42c51-kube-api-access-tnmc4\") pod \"maas-api-5846b87474-2rgtl\" (UID: \"28982306-24aa-4afa-926e-05866cc42c51\") " pod="opendatahub/maas-api-5846b87474-2rgtl" Jul 06 19:57:48.384785 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.384662 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28982306-24aa-4afa-926e-05866cc42c51-openshift-service-ca\") pod \"maas-api-5846b87474-2rgtl\" (UID: \"28982306-24aa-4afa-926e-05866cc42c51\") " pod="opendatahub/maas-api-5846b87474-2rgtl" Jul 06 19:57:48.385350 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.385303 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28982306-24aa-4afa-926e-05866cc42c51-openshift-service-ca\") pod \"maas-api-5846b87474-2rgtl\" (UID: \"28982306-24aa-4afa-926e-05866cc42c51\") " pod="opendatahub/maas-api-5846b87474-2rgtl" Jul 06 19:57:48.387042 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.387014 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28982306-24aa-4afa-926e-05866cc42c51-maas-api-tls\") pod \"maas-api-5846b87474-2rgtl\" (UID: \"28982306-24aa-4afa-926e-05866cc42c51\") " pod="opendatahub/maas-api-5846b87474-2rgtl" Jul 06 19:57:48.393930 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.393904 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnmc4\" (UniqueName: \"kubernetes.io/projected/28982306-24aa-4afa-926e-05866cc42c51-kube-api-access-tnmc4\") pod \"maas-api-5846b87474-2rgtl\" (UID: \"28982306-24aa-4afa-926e-05866cc42c51\") " pod="opendatahub/maas-api-5846b87474-2rgtl" Jul 06 19:57:48.535226 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.535188 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5846b87474-2rgtl" Jul 06 19:57:48.679717 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.679688 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5846b87474-2rgtl"] Jul 06 19:57:48.681801 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:57:48.681777 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28982306_24aa_4afa_926e_05866cc42c51.slice/crio-d7c7d4154d1a603f9f657e080509095d58ef5689dc1c7642bb6ae13095a8ee64 WatchSource:0}: Error finding container d7c7d4154d1a603f9f657e080509095d58ef5689dc1c7642bb6ae13095a8ee64: Status 404 returned error can't find the container with id d7c7d4154d1a603f9f657e080509095d58ef5689dc1c7642bb6ae13095a8ee64 Jul 06 19:57:48.717469 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.717438 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5846b87474-2rgtl" event={"ID":"28982306-24aa-4afa-926e-05866cc42c51","Type":"ContainerStarted","Data":"d7c7d4154d1a603f9f657e080509095d58ef5689dc1c7642bb6ae13095a8ee64"} Jul 06 19:57:48.846631 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.846552 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-kv56k"] Jul 06 19:57:48.850685 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.850661 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-kv56k" Jul 06 19:57:48.853111 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.853088 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-lvhhp\"" Jul 06 19:57:48.853217 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.853087 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 19:57:48.854486 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.854463 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-44wkr"] Jul 06 19:57:48.858023 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.858006 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-44wkr" Jul 06 19:57:48.862060 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.862036 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-kv56k"] Jul 06 19:57:48.868475 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.868451 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-44wkr"] Jul 06 19:57:48.888894 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.888863 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfhfs\" (UniqueName: \"kubernetes.io/projected/9137adb9-e557-4709-9f69-ab31edf80048-kube-api-access-bfhfs\") pod \"payload-pre-processing-7964457696-kv56k\" (UID: \"9137adb9-e557-4709-9f69-ab31edf80048\") " pod="openshift-ingress/payload-pre-processing-7964457696-kv56k" Jul 06 19:57:48.889058 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.889012 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9137adb9-e557-4709-9f69-ab31edf80048-plugins-config-volume\") pod \"payload-pre-processing-7964457696-kv56k\" (UID: \"9137adb9-e557-4709-9f69-ab31edf80048\") " pod="openshift-ingress/payload-pre-processing-7964457696-kv56k" Jul 06 19:57:48.990598 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.990551 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5f3b493-9b0f-44d8-aba5-257b0c46f540-plugins-config-volume\") pod \"payload-processing-58f86c7798-44wkr\" (UID: \"a5f3b493-9b0f-44d8-aba5-257b0c46f540\") " pod="openshift-ingress/payload-processing-58f86c7798-44wkr" Jul 06 19:57:48.990598 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.990589 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6hbm\" (UniqueName: \"kubernetes.io/projected/a5f3b493-9b0f-44d8-aba5-257b0c46f540-kube-api-access-b6hbm\") pod \"payload-processing-58f86c7798-44wkr\" (UID: \"a5f3b493-9b0f-44d8-aba5-257b0c46f540\") " pod="openshift-ingress/payload-processing-58f86c7798-44wkr" Jul 06 19:57:48.990851 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.990693 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfhfs\" (UniqueName: \"kubernetes.io/projected/9137adb9-e557-4709-9f69-ab31edf80048-kube-api-access-bfhfs\") pod \"payload-pre-processing-7964457696-kv56k\" (UID: \"9137adb9-e557-4709-9f69-ab31edf80048\") " pod="openshift-ingress/payload-pre-processing-7964457696-kv56k" Jul 06 19:57:48.990851 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.990746 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9137adb9-e557-4709-9f69-ab31edf80048-plugins-config-volume\") pod \"payload-pre-processing-7964457696-kv56k\" (UID: \"9137adb9-e557-4709-9f69-ab31edf80048\") " pod="openshift-ingress/payload-pre-processing-7964457696-kv56k" Jul 06 19:57:48.991400 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.991380 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9137adb9-e557-4709-9f69-ab31edf80048-plugins-config-volume\") pod \"payload-pre-processing-7964457696-kv56k\" (UID: \"9137adb9-e557-4709-9f69-ab31edf80048\") " pod="openshift-ingress/payload-pre-processing-7964457696-kv56k" Jul 06 19:57:48.999135 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:48.999112 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfhfs\" (UniqueName: \"kubernetes.io/projected/9137adb9-e557-4709-9f69-ab31edf80048-kube-api-access-bfhfs\") pod \"payload-pre-processing-7964457696-kv56k\" (UID: \"9137adb9-e557-4709-9f69-ab31edf80048\") " pod="openshift-ingress/payload-pre-processing-7964457696-kv56k" Jul 06 19:57:49.091298 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:49.091262 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5f3b493-9b0f-44d8-aba5-257b0c46f540-plugins-config-volume\") pod \"payload-processing-58f86c7798-44wkr\" (UID: \"a5f3b493-9b0f-44d8-aba5-257b0c46f540\") " pod="openshift-ingress/payload-processing-58f86c7798-44wkr" Jul 06 19:57:49.091298 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:49.091299 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6hbm\" (UniqueName: \"kubernetes.io/projected/a5f3b493-9b0f-44d8-aba5-257b0c46f540-kube-api-access-b6hbm\") pod \"payload-processing-58f86c7798-44wkr\" (UID: \"a5f3b493-9b0f-44d8-aba5-257b0c46f540\") " pod="openshift-ingress/payload-processing-58f86c7798-44wkr" Jul 06 19:57:49.092015 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:49.091995 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a5f3b493-9b0f-44d8-aba5-257b0c46f540-plugins-config-volume\") pod \"payload-processing-58f86c7798-44wkr\" (UID: \"a5f3b493-9b0f-44d8-aba5-257b0c46f540\") " pod="openshift-ingress/payload-processing-58f86c7798-44wkr" Jul 06 19:57:49.100241 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:49.100180 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6hbm\" (UniqueName: \"kubernetes.io/projected/a5f3b493-9b0f-44d8-aba5-257b0c46f540-kube-api-access-b6hbm\") pod \"payload-processing-58f86c7798-44wkr\" (UID: \"a5f3b493-9b0f-44d8-aba5-257b0c46f540\") " pod="openshift-ingress/payload-processing-58f86c7798-44wkr" Jul 06 19:57:49.163679 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:49.163645 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-kv56k" Jul 06 19:57:49.172437 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:49.172413 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-44wkr" Jul 06 19:57:49.304154 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:49.304123 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-kv56k"] Jul 06 19:57:49.305911 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:57:49.305880 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9137adb9_e557_4709_9f69_ab31edf80048.slice/crio-cbc54551ebfbf5f9c118bbb3866d50bd89483f72ffeabe651cb1bc1b614ba2c0 WatchSource:0}: Error finding container cbc54551ebfbf5f9c118bbb3866d50bd89483f72ffeabe651cb1bc1b614ba2c0: Status 404 returned error can't find the container with id cbc54551ebfbf5f9c118bbb3866d50bd89483f72ffeabe651cb1bc1b614ba2c0 Jul 06 19:57:49.332688 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:49.332666 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-44wkr"] Jul 06 19:57:49.334254 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:57:49.334214 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5f3b493_9b0f_44d8_aba5_257b0c46f540.slice/crio-fb3cef81ef318ebb1ea91eb979142b69c29b079fcbe6c3d2fae4cccaf17ea57d WatchSource:0}: Error finding container fb3cef81ef318ebb1ea91eb979142b69c29b079fcbe6c3d2fae4cccaf17ea57d: Status 404 returned error can't find the container with id fb3cef81ef318ebb1ea91eb979142b69c29b079fcbe6c3d2fae4cccaf17ea57d Jul 06 19:57:49.724772 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:49.724728 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-44wkr" event={"ID":"a5f3b493-9b0f-44d8-aba5-257b0c46f540","Type":"ContainerStarted","Data":"fb3cef81ef318ebb1ea91eb979142b69c29b079fcbe6c3d2fae4cccaf17ea57d"} Jul 06 19:57:49.726098 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:49.726070 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-kv56k" event={"ID":"9137adb9-e557-4709-9f69-ab31edf80048","Type":"ContainerStarted","Data":"cbc54551ebfbf5f9c118bbb3866d50bd89483f72ffeabe651cb1bc1b614ba2c0"} Jul 06 19:57:51.738404 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:51.738364 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5846b87474-2rgtl" event={"ID":"28982306-24aa-4afa-926e-05866cc42c51","Type":"ContainerStarted","Data":"b04474b9a961c6e59deb6c1423547ab713720ffc87bf87960e71a5dee6b15b6f"} Jul 06 19:57:51.738859 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:51.738527 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-5846b87474-2rgtl" Jul 06 19:57:51.739951 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:51.739926 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-44wkr" event={"ID":"a5f3b493-9b0f-44d8-aba5-257b0c46f540","Type":"ContainerStarted","Data":"f01692e8d306860c88e93be103854da3ef22710cced5d42a15d6a79b1baa56be"} Jul 06 19:57:51.740070 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:51.739980 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-44wkr" Jul 06 19:57:51.741297 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:51.741278 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-kv56k" event={"ID":"9137adb9-e557-4709-9f69-ab31edf80048","Type":"ContainerStarted","Data":"1d760f3334b864f5acec26fc271e38ae78acc07be3483518883f9126777ffdc8"} Jul 06 19:57:51.741440 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:51.741374 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-kv56k" Jul 06 19:57:51.760227 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:51.760160 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-5846b87474-2rgtl" podStartSLOduration=1.349070308 podStartE2EDuration="3.760140747s" totalImagesPullingTime="2.411070439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:57:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:57:51.756553891 +0000 UTC m=+571.203790621" watchObservedRunningTime="2026-07-06 19:57:51.760140747 +0000 UTC m=+571.207377465" Jul 06 19:57:51.781393 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:51.781311 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-kv56k" podStartSLOduration=1.992663017 podStartE2EDuration="3.781289089s" totalImagesPullingTime="1.788626072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:57:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:57:51.780764864 +0000 UTC m=+571.228001586" watchObservedRunningTime="2026-07-06 19:57:51.781289089 +0000 UTC m=+571.228525808" Jul 06 19:57:51.814266 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:51.814204 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-44wkr" podStartSLOduration=2.056083566 podStartE2EDuration="3.814183033s" totalImagesPullingTime="1.758099467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:57:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:57:51.814006899 +0000 UTC m=+571.261243659" watchObservedRunningTime="2026-07-06 19:57:51.814183033 +0000 UTC m=+571.261419754" Jul 06 19:57:56.771458 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:56.771411 2581 generic.go:356] "Generic (PLEG): container finished" podID="266f87bd-dd56-4772-8d0d-4cfe66e673fb" containerID="84f8ffc7446b2da28b92eea5c3868b19f42a66161cd3233bd6c92aebcbdc2d2f" exitCode=0 Jul 06 19:57:56.771875 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:56.771486 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" event={"ID":"266f87bd-dd56-4772-8d0d-4cfe66e673fb","Type":"ContainerDied","Data":"84f8ffc7446b2da28b92eea5c3868b19f42a66161cd3233bd6c92aebcbdc2d2f"} Jul 06 19:57:56.771924 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:56.771884 2581 scope.go:117] "RemoveContainer" containerID="84f8ffc7446b2da28b92eea5c3868b19f42a66161cd3233bd6c92aebcbdc2d2f" Jul 06 19:57:56.772896 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:56.772864 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 19:57:56.773007 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:56.772950 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 19:57:56.773049 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:56.772998 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 19:57:57.764653 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:57.764613 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-5846b87474-2rgtl" Jul 06 19:57:57.778303 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:57.778271 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" event={"ID":"266f87bd-dd56-4772-8d0d-4cfe66e673fb","Type":"ContainerStarted","Data":"d1e956be84a4bf63f7bbfe2d36706145a20d45e042d1bfb3226781c2afb12a2f"} Jul 06 19:57:57.858779 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:57.858745 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:57:57.859169 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:57.859079 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-lc6bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 06 19:57:57.859356 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:57.859304 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" podUID="266f87bd-dd56-4772-8d0d-4cfe66e673fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 06 19:57:58.851165 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:58.851122 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-lc6bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 06 19:57:58.851690 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:58.851188 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" podUID="266f87bd-dd56-4772-8d0d-4cfe66e673fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 06 19:57:59.851708 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:59.851666 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-lc6bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 06 19:57:59.852190 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:57:59.851735 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" podUID="266f87bd-dd56-4772-8d0d-4cfe66e673fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 06 19:58:00.851762 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:00.851725 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-lc6bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body= Jul 06 19:58:00.852380 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:00.852339 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" podUID="266f87bd-dd56-4772-8d0d-4cfe66e673fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" Jul 06 19:58:01.854429 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:01.854398 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:58:01.854864 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:01.854649 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:58:01.855566 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:01.855545 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-lc6bh" Jul 06 19:58:02.752150 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:02.752118 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-44wkr" Jul 06 19:58:02.752381 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:02.752190 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-kv56k" Jul 06 19:58:12.402641 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.402608 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq"] Jul 06 19:58:12.407912 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.407882 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:12.411459 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.411433 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 19:58:12.411594 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.411433 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6wr7c\"" Jul 06 19:58:12.412587 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.412566 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 19:58:12.412736 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.412720 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 06 19:58:12.418567 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.418534 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq"] Jul 06 19:58:12.513110 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.513071 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6db7aaf-1c96-48ad-94b5-472c2b6b36b1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq\" (UID: \"c6db7aaf-1c96-48ad-94b5-472c2b6b36b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:12.513350 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.513162 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfhg2\" (UniqueName: \"kubernetes.io/projected/c6db7aaf-1c96-48ad-94b5-472c2b6b36b1-kube-api-access-tfhg2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq\" (UID: \"c6db7aaf-1c96-48ad-94b5-472c2b6b36b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:12.513350 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.513214 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6db7aaf-1c96-48ad-94b5-472c2b6b36b1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq\" (UID: \"c6db7aaf-1c96-48ad-94b5-472c2b6b36b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:12.513350 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.513257 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6db7aaf-1c96-48ad-94b5-472c2b6b36b1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq\" (UID: \"c6db7aaf-1c96-48ad-94b5-472c2b6b36b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:12.513509 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.513359 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6db7aaf-1c96-48ad-94b5-472c2b6b36b1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq\" (UID: \"c6db7aaf-1c96-48ad-94b5-472c2b6b36b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:12.513509 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.513389 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6db7aaf-1c96-48ad-94b5-472c2b6b36b1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq\" (UID: \"c6db7aaf-1c96-48ad-94b5-472c2b6b36b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:12.614728 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.614695 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfhg2\" (UniqueName: \"kubernetes.io/projected/c6db7aaf-1c96-48ad-94b5-472c2b6b36b1-kube-api-access-tfhg2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq\" (UID: \"c6db7aaf-1c96-48ad-94b5-472c2b6b36b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:12.614905 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.614750 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6db7aaf-1c96-48ad-94b5-472c2b6b36b1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq\" (UID: \"c6db7aaf-1c96-48ad-94b5-472c2b6b36b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:12.614905 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.614865 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6db7aaf-1c96-48ad-94b5-472c2b6b36b1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq\" (UID: \"c6db7aaf-1c96-48ad-94b5-472c2b6b36b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:12.615021 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.615005 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6db7aaf-1c96-48ad-94b5-472c2b6b36b1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq\" (UID: \"c6db7aaf-1c96-48ad-94b5-472c2b6b36b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:12.615072 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.615036 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6db7aaf-1c96-48ad-94b5-472c2b6b36b1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq\" (UID: \"c6db7aaf-1c96-48ad-94b5-472c2b6b36b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:12.615072 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.615053 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6db7aaf-1c96-48ad-94b5-472c2b6b36b1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq\" (UID: \"c6db7aaf-1c96-48ad-94b5-472c2b6b36b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:12.615151 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.615105 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6db7aaf-1c96-48ad-94b5-472c2b6b36b1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq\" (UID: \"c6db7aaf-1c96-48ad-94b5-472c2b6b36b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:12.615352 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.615331 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6db7aaf-1c96-48ad-94b5-472c2b6b36b1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq\" (UID: \"c6db7aaf-1c96-48ad-94b5-472c2b6b36b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:12.615461 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.615443 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6db7aaf-1c96-48ad-94b5-472c2b6b36b1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq\" (UID: \"c6db7aaf-1c96-48ad-94b5-472c2b6b36b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:12.617186 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.617162 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6db7aaf-1c96-48ad-94b5-472c2b6b36b1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq\" (UID: \"c6db7aaf-1c96-48ad-94b5-472c2b6b36b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:12.617365 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.617349 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6db7aaf-1c96-48ad-94b5-472c2b6b36b1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq\" (UID: \"c6db7aaf-1c96-48ad-94b5-472c2b6b36b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:12.625208 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.625185 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfhg2\" (UniqueName: \"kubernetes.io/projected/c6db7aaf-1c96-48ad-94b5-472c2b6b36b1-kube-api-access-tfhg2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq\" (UID: \"c6db7aaf-1c96-48ad-94b5-472c2b6b36b1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:12.720235 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.720160 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:12.861690 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:12.861668 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq"] Jul 06 19:58:12.864721 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:58:12.864681 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6db7aaf_1c96_48ad_94b5_472c2b6b36b1.slice/crio-f8b720d7c43fb43af32181a7810dac5acf103c9fad33081f8d9fbbbb4772cf22 WatchSource:0}: Error finding container f8b720d7c43fb43af32181a7810dac5acf103c9fad33081f8d9fbbbb4772cf22: Status 404 returned error can't find the container with id f8b720d7c43fb43af32181a7810dac5acf103c9fad33081f8d9fbbbb4772cf22 Jul 06 19:58:13.859207 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:13.859167 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" event={"ID":"c6db7aaf-1c96-48ad-94b5-472c2b6b36b1","Type":"ContainerStarted","Data":"f8b720d7c43fb43af32181a7810dac5acf103c9fad33081f8d9fbbbb4772cf22"} Jul 06 19:58:14.865171 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:14.865137 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" event={"ID":"c6db7aaf-1c96-48ad-94b5-472c2b6b36b1","Type":"ContainerStarted","Data":"9acff5742298e726950292d159baee2b588b7cb28b5eaf0f603b6820f919680d"} Jul 06 19:58:14.886296 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:14.886248 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" podStartSLOduration=1.6921564660000001 podStartE2EDuration="2.886234411s" totalImagesPullingTime="1.194077945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:58:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:58:14.884865607 +0000 UTC m=+594.332102325" watchObservedRunningTime="2026-07-06 19:58:14.886234411 +0000 UTC m=+594.333471128" Jul 06 19:58:21.014706 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:21.014674 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/2.log" Jul 06 19:58:21.016414 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:21.016392 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/2.log" Jul 06 19:58:22.720867 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:22.720833 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:22.721389 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:22.720901 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:22.733140 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:22.733111 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:22.916072 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:22.916030 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq" Jul 06 19:58:42.898006 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:42.897970 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd"] Jul 06 19:58:42.903314 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:42.903292 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:42.906150 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:42.906129 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 06 19:58:42.920995 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:42.920958 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd"] Jul 06 19:58:43.006564 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:43.006530 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4095e67f-7cd4-45bf-8b4e-860ae54e8f9c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd\" (UID: \"4095e67f-7cd4-45bf-8b4e-860ae54e8f9c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:43.006751 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:43.006570 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4095e67f-7cd4-45bf-8b4e-860ae54e8f9c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd\" (UID: \"4095e67f-7cd4-45bf-8b4e-860ae54e8f9c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:43.006751 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:43.006667 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4095e67f-7cd4-45bf-8b4e-860ae54e8f9c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd\" (UID: \"4095e67f-7cd4-45bf-8b4e-860ae54e8f9c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:43.006751 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:43.006727 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4095e67f-7cd4-45bf-8b4e-860ae54e8f9c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd\" (UID: \"4095e67f-7cd4-45bf-8b4e-860ae54e8f9c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:43.006883 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:43.006755 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4dl8\" (UniqueName: \"kubernetes.io/projected/4095e67f-7cd4-45bf-8b4e-860ae54e8f9c-kube-api-access-d4dl8\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd\" (UID: \"4095e67f-7cd4-45bf-8b4e-860ae54e8f9c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:43.006883 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:43.006856 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4095e67f-7cd4-45bf-8b4e-860ae54e8f9c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd\" (UID: \"4095e67f-7cd4-45bf-8b4e-860ae54e8f9c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:43.107805 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:43.107776 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4095e67f-7cd4-45bf-8b4e-860ae54e8f9c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd\" (UID: \"4095e67f-7cd4-45bf-8b4e-860ae54e8f9c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:43.107984 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:43.107826 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4095e67f-7cd4-45bf-8b4e-860ae54e8f9c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd\" (UID: \"4095e67f-7cd4-45bf-8b4e-860ae54e8f9c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:43.107984 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:43.107850 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4095e67f-7cd4-45bf-8b4e-860ae54e8f9c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd\" (UID: \"4095e67f-7cd4-45bf-8b4e-860ae54e8f9c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:43.107984 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:43.107884 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4dl8\" (UniqueName: \"kubernetes.io/projected/4095e67f-7cd4-45bf-8b4e-860ae54e8f9c-kube-api-access-d4dl8\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd\" (UID: \"4095e67f-7cd4-45bf-8b4e-860ae54e8f9c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:43.107984 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:43.107952 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4095e67f-7cd4-45bf-8b4e-860ae54e8f9c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd\" (UID: \"4095e67f-7cd4-45bf-8b4e-860ae54e8f9c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:43.108197 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:43.108002 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4095e67f-7cd4-45bf-8b4e-860ae54e8f9c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd\" (UID: \"4095e67f-7cd4-45bf-8b4e-860ae54e8f9c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:43.108197 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:43.108177 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4095e67f-7cd4-45bf-8b4e-860ae54e8f9c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd\" (UID: \"4095e67f-7cd4-45bf-8b4e-860ae54e8f9c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:43.108303 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:43.108226 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4095e67f-7cd4-45bf-8b4e-860ae54e8f9c-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd\" (UID: \"4095e67f-7cd4-45bf-8b4e-860ae54e8f9c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:43.108383 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:43.108357 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4095e67f-7cd4-45bf-8b4e-860ae54e8f9c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd\" (UID: \"4095e67f-7cd4-45bf-8b4e-860ae54e8f9c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:43.110135 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:43.110107 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4095e67f-7cd4-45bf-8b4e-860ae54e8f9c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd\" (UID: \"4095e67f-7cd4-45bf-8b4e-860ae54e8f9c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:43.110363 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:43.110347 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4095e67f-7cd4-45bf-8b4e-860ae54e8f9c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd\" (UID: \"4095e67f-7cd4-45bf-8b4e-860ae54e8f9c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:43.116157 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:43.116133 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4dl8\" (UniqueName: \"kubernetes.io/projected/4095e67f-7cd4-45bf-8b4e-860ae54e8f9c-kube-api-access-d4dl8\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd\" (UID: \"4095e67f-7cd4-45bf-8b4e-860ae54e8f9c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:43.215570 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:43.215460 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:43.351489 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:43.351464 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd"] Jul 06 19:58:43.353628 ip-10-0-130-172 kubenswrapper[2581]: W0706 19:58:43.353594 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4095e67f_7cd4_45bf_8b4e_860ae54e8f9c.slice/crio-8aad57bd4edccb6443e8c8472ca64281cfd78dc740d5d48507154f44a481ccbd WatchSource:0}: Error finding container 8aad57bd4edccb6443e8c8472ca64281cfd78dc740d5d48507154f44a481ccbd: Status 404 returned error can't find the container with id 8aad57bd4edccb6443e8c8472ca64281cfd78dc740d5d48507154f44a481ccbd Jul 06 19:58:43.355038 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:43.355018 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:58:44.011969 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:44.011938 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" event={"ID":"4095e67f-7cd4-45bf-8b4e-860ae54e8f9c","Type":"ContainerStarted","Data":"7f7ea3dfad40136978e08f22195a25138994c04c10217d8dadc1cb97d1d9bd52"} Jul 06 19:58:44.012381 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:44.011986 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" event={"ID":"4095e67f-7cd4-45bf-8b4e-860ae54e8f9c","Type":"ContainerStarted","Data":"8aad57bd4edccb6443e8c8472ca64281cfd78dc740d5d48507154f44a481ccbd"} Jul 06 19:58:44.031539 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:44.031494 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" podStartSLOduration=1.820268559 podStartE2EDuration="2.031479714s" totalImagesPullingTime="211.211155ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:58:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:58:44.029607327 +0000 UTC m=+623.476844046" watchObservedRunningTime="2026-07-06 19:58:44.031479714 +0000 UTC m=+623.478716432" Jul 06 19:58:53.215747 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:53.215706 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:53.216635 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:53.215819 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:53.228522 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:53.228498 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 19:58:54.072243 ip-10-0-130-172 kubenswrapper[2581]: I0706 19:58:54.072218 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd" Jul 06 20:00:00.140428 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:00.140393 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722800-f5mtb"] Jul 06 20:00:00.143979 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:00.143959 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722800-f5mtb" Jul 06 20:00:00.146288 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:00.146269 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-qkn72\"" Jul 06 20:00:00.152691 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:00.152668 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722800-f5mtb"] Jul 06 20:00:00.239457 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:00.239423 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cq7h\" (UniqueName: \"kubernetes.io/projected/12c5da94-35d7-4262-8cf1-6cb83ba4b60f-kube-api-access-8cq7h\") pod \"maas-api-key-cleanup-29722800-f5mtb\" (UID: \"12c5da94-35d7-4262-8cf1-6cb83ba4b60f\") " pod="opendatahub/maas-api-key-cleanup-29722800-f5mtb" Jul 06 20:00:00.340282 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:00.340245 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cq7h\" (UniqueName: \"kubernetes.io/projected/12c5da94-35d7-4262-8cf1-6cb83ba4b60f-kube-api-access-8cq7h\") pod \"maas-api-key-cleanup-29722800-f5mtb\" (UID: \"12c5da94-35d7-4262-8cf1-6cb83ba4b60f\") " pod="opendatahub/maas-api-key-cleanup-29722800-f5mtb" Jul 06 20:00:00.348619 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:00.348593 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cq7h\" (UniqueName: \"kubernetes.io/projected/12c5da94-35d7-4262-8cf1-6cb83ba4b60f-kube-api-access-8cq7h\") pod \"maas-api-key-cleanup-29722800-f5mtb\" (UID: \"12c5da94-35d7-4262-8cf1-6cb83ba4b60f\") " pod="opendatahub/maas-api-key-cleanup-29722800-f5mtb" Jul 06 20:00:00.456280 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:00.456192 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722800-f5mtb" Jul 06 20:00:00.593478 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:00.593452 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722800-f5mtb"] Jul 06 20:00:00.596258 ip-10-0-130-172 kubenswrapper[2581]: W0706 20:00:00.596219 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12c5da94_35d7_4262_8cf1_6cb83ba4b60f.slice/crio-a7d0f8c39be096a161b370e23444279769baadf4704b7505b41a30dd44ea1e92 WatchSource:0}: Error finding container a7d0f8c39be096a161b370e23444279769baadf4704b7505b41a30dd44ea1e92: Status 404 returned error can't find the container with id a7d0f8c39be096a161b370e23444279769baadf4704b7505b41a30dd44ea1e92 Jul 06 20:00:01.385829 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:01.385794 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722800-f5mtb" event={"ID":"12c5da94-35d7-4262-8cf1-6cb83ba4b60f","Type":"ContainerStarted","Data":"a7d0f8c39be096a161b370e23444279769baadf4704b7505b41a30dd44ea1e92"} Jul 06 20:00:08.151598 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:08.151561 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-779449988c-44d5k"] Jul 06 20:00:08.152102 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:08.151794 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-779449988c-44d5k" podUID="33bae915-3dbf-4a90-8d7b-f7980a8b7e59" containerName="authorino" containerID="cri-o://83f0263c1c7da03774d39b91dba3d2ed38e3d3967dcaa3482c7b9cc82cbc8b24" gracePeriod=30 Jul 06 20:00:08.407533 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:08.407465 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-779449988c-44d5k" Jul 06 20:00:08.419903 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:08.419872 2581 generic.go:356] "Generic (PLEG): container finished" podID="33bae915-3dbf-4a90-8d7b-f7980a8b7e59" containerID="83f0263c1c7da03774d39b91dba3d2ed38e3d3967dcaa3482c7b9cc82cbc8b24" exitCode=0 Jul 06 20:00:08.420051 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:08.419911 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-779449988c-44d5k" Jul 06 20:00:08.420051 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:08.419928 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-779449988c-44d5k" event={"ID":"33bae915-3dbf-4a90-8d7b-f7980a8b7e59","Type":"ContainerDied","Data":"83f0263c1c7da03774d39b91dba3d2ed38e3d3967dcaa3482c7b9cc82cbc8b24"} Jul 06 20:00:08.420051 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:08.419978 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-779449988c-44d5k" event={"ID":"33bae915-3dbf-4a90-8d7b-f7980a8b7e59","Type":"ContainerDied","Data":"663b3ea8c9bc3af70f96aa95f6504827f041ad2f5e909b1a61804aaf3873425d"} Jul 06 20:00:08.420051 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:08.419997 2581 scope.go:117] "RemoveContainer" containerID="83f0263c1c7da03774d39b91dba3d2ed38e3d3967dcaa3482c7b9cc82cbc8b24" Jul 06 20:00:08.430635 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:08.430610 2581 scope.go:117] "RemoveContainer" containerID="83f0263c1c7da03774d39b91dba3d2ed38e3d3967dcaa3482c7b9cc82cbc8b24" Jul 06 20:00:08.430971 ip-10-0-130-172 kubenswrapper[2581]: E0706 20:00:08.430948 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83f0263c1c7da03774d39b91dba3d2ed38e3d3967dcaa3482c7b9cc82cbc8b24\": container with ID starting with 83f0263c1c7da03774d39b91dba3d2ed38e3d3967dcaa3482c7b9cc82cbc8b24 not found: ID does not exist" containerID="83f0263c1c7da03774d39b91dba3d2ed38e3d3967dcaa3482c7b9cc82cbc8b24" Jul 06 20:00:08.431050 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:08.430980 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83f0263c1c7da03774d39b91dba3d2ed38e3d3967dcaa3482c7b9cc82cbc8b24"} err="failed to get container status \"83f0263c1c7da03774d39b91dba3d2ed38e3d3967dcaa3482c7b9cc82cbc8b24\": rpc error: code = NotFound desc = could not find container \"83f0263c1c7da03774d39b91dba3d2ed38e3d3967dcaa3482c7b9cc82cbc8b24\": container with ID starting with 83f0263c1c7da03774d39b91dba3d2ed38e3d3967dcaa3482c7b9cc82cbc8b24 not found: ID does not exist" Jul 06 20:00:08.517188 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:08.517149 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/33bae915-3dbf-4a90-8d7b-f7980a8b7e59-tls-cert\") pod \"33bae915-3dbf-4a90-8d7b-f7980a8b7e59\" (UID: \"33bae915-3dbf-4a90-8d7b-f7980a8b7e59\") " Jul 06 20:00:08.517427 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:08.517246 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bz5x\" (UniqueName: \"kubernetes.io/projected/33bae915-3dbf-4a90-8d7b-f7980a8b7e59-kube-api-access-7bz5x\") pod \"33bae915-3dbf-4a90-8d7b-f7980a8b7e59\" (UID: \"33bae915-3dbf-4a90-8d7b-f7980a8b7e59\") " Jul 06 20:00:08.519145 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:08.519122 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33bae915-3dbf-4a90-8d7b-f7980a8b7e59-kube-api-access-7bz5x" (OuterVolumeSpecName: "kube-api-access-7bz5x") pod "33bae915-3dbf-4a90-8d7b-f7980a8b7e59" (UID: "33bae915-3dbf-4a90-8d7b-f7980a8b7e59"). InnerVolumeSpecName "kube-api-access-7bz5x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:00:08.529425 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:08.529390 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33bae915-3dbf-4a90-8d7b-f7980a8b7e59-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "33bae915-3dbf-4a90-8d7b-f7980a8b7e59" (UID: "33bae915-3dbf-4a90-8d7b-f7980a8b7e59"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:00:08.618063 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:08.618031 2581 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/33bae915-3dbf-4a90-8d7b-f7980a8b7e59-tls-cert\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:00:08.618063 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:08.618057 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bz5x\" (UniqueName: \"kubernetes.io/projected/33bae915-3dbf-4a90-8d7b-f7980a8b7e59-kube-api-access-7bz5x\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:00:08.751631 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:08.751603 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-779449988c-44d5k"] Jul 06 20:00:08.756692 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:08.756669 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-779449988c-44d5k"] Jul 06 20:00:09.094791 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:09.094754 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33bae915-3dbf-4a90-8d7b-f7980a8b7e59" path="/var/lib/kubelet/pods/33bae915-3dbf-4a90-8d7b-f7980a8b7e59/volumes" Jul 06 20:00:22.490556 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:22.490522 2581 generic.go:356] "Generic (PLEG): container finished" podID="12c5da94-35d7-4262-8cf1-6cb83ba4b60f" containerID="94ae940bbb3fc844b9eefe8284ba8d666632a3cb285e87a1d7f343a87b152849" exitCode=0 Jul 06 20:00:22.490901 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:22.490566 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722800-f5mtb" event={"ID":"12c5da94-35d7-4262-8cf1-6cb83ba4b60f","Type":"ContainerDied","Data":"94ae940bbb3fc844b9eefe8284ba8d666632a3cb285e87a1d7f343a87b152849"} Jul 06 20:00:23.641747 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:23.641711 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722800-f5mtb" Jul 06 20:00:23.669186 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:23.669146 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cq7h\" (UniqueName: \"kubernetes.io/projected/12c5da94-35d7-4262-8cf1-6cb83ba4b60f-kube-api-access-8cq7h\") pod \"12c5da94-35d7-4262-8cf1-6cb83ba4b60f\" (UID: \"12c5da94-35d7-4262-8cf1-6cb83ba4b60f\") " Jul 06 20:00:23.671248 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:23.671212 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12c5da94-35d7-4262-8cf1-6cb83ba4b60f-kube-api-access-8cq7h" (OuterVolumeSpecName: "kube-api-access-8cq7h") pod "12c5da94-35d7-4262-8cf1-6cb83ba4b60f" (UID: "12c5da94-35d7-4262-8cf1-6cb83ba4b60f"). InnerVolumeSpecName "kube-api-access-8cq7h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:00:23.770756 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:23.770723 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cq7h\" (UniqueName: \"kubernetes.io/projected/12c5da94-35d7-4262-8cf1-6cb83ba4b60f-kube-api-access-8cq7h\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:00:24.501952 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:24.501922 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722800-f5mtb" Jul 06 20:00:24.501952 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:24.501952 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722800-f5mtb" event={"ID":"12c5da94-35d7-4262-8cf1-6cb83ba4b60f","Type":"ContainerDied","Data":"a7d0f8c39be096a161b370e23444279769baadf4704b7505b41a30dd44ea1e92"} Jul 06 20:00:24.502211 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:00:24.502001 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a7d0f8c39be096a161b370e23444279769baadf4704b7505b41a30dd44ea1e92" Jul 06 20:01:18.706926 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:18.706882 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7588cc9ccd-7m9ws"] Jul 06 20:01:18.707501 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:18.707122 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-7588cc9ccd-7m9ws" podUID="c2dbcb90-1a1e-4d77-89f6-e408cbba8728" containerName="manager" containerID="cri-o://6469161b3408f6311e3e475a599e9813687634849322da8ff5e632921bba612a" gracePeriod=30 Jul 06 20:01:18.962401 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:18.962338 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7588cc9ccd-7m9ws" Jul 06 20:01:19.104886 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:19.104844 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6qc5\" (UniqueName: \"kubernetes.io/projected/c2dbcb90-1a1e-4d77-89f6-e408cbba8728-kube-api-access-w6qc5\") pod \"c2dbcb90-1a1e-4d77-89f6-e408cbba8728\" (UID: \"c2dbcb90-1a1e-4d77-89f6-e408cbba8728\") " Jul 06 20:01:19.105042 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:19.104936 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2dbcb90-1a1e-4d77-89f6-e408cbba8728-webhook-cert\") pod \"c2dbcb90-1a1e-4d77-89f6-e408cbba8728\" (UID: \"c2dbcb90-1a1e-4d77-89f6-e408cbba8728\") " Jul 06 20:01:19.106906 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:19.106880 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2dbcb90-1a1e-4d77-89f6-e408cbba8728-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c2dbcb90-1a1e-4d77-89f6-e408cbba8728" (UID: "c2dbcb90-1a1e-4d77-89f6-e408cbba8728"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 20:01:19.107015 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:19.106981 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2dbcb90-1a1e-4d77-89f6-e408cbba8728-kube-api-access-w6qc5" (OuterVolumeSpecName: "kube-api-access-w6qc5") pod "c2dbcb90-1a1e-4d77-89f6-e408cbba8728" (UID: "c2dbcb90-1a1e-4d77-89f6-e408cbba8728"). InnerVolumeSpecName "kube-api-access-w6qc5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:01:19.205872 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:19.205824 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6qc5\" (UniqueName: \"kubernetes.io/projected/c2dbcb90-1a1e-4d77-89f6-e408cbba8728-kube-api-access-w6qc5\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:01:19.205872 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:19.205866 2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2dbcb90-1a1e-4d77-89f6-e408cbba8728-webhook-cert\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:01:19.781035 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:19.780998 2581 generic.go:356] "Generic (PLEG): container finished" podID="c2dbcb90-1a1e-4d77-89f6-e408cbba8728" containerID="6469161b3408f6311e3e475a599e9813687634849322da8ff5e632921bba612a" exitCode=0 Jul 06 20:01:19.781467 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:19.781055 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7588cc9ccd-7m9ws" Jul 06 20:01:19.781467 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:19.781086 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7588cc9ccd-7m9ws" event={"ID":"c2dbcb90-1a1e-4d77-89f6-e408cbba8728","Type":"ContainerDied","Data":"6469161b3408f6311e3e475a599e9813687634849322da8ff5e632921bba612a"} Jul 06 20:01:19.781467 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:19.781141 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7588cc9ccd-7m9ws" event={"ID":"c2dbcb90-1a1e-4d77-89f6-e408cbba8728","Type":"ContainerDied","Data":"83c022cfbad4f7487c6e9e0e9938e2ae36cdd014ec8f88f6181061b89c598c0d"} Jul 06 20:01:19.781467 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:19.781163 2581 scope.go:117] "RemoveContainer" containerID="6469161b3408f6311e3e475a599e9813687634849322da8ff5e632921bba612a" Jul 06 20:01:19.790972 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:19.790955 2581 scope.go:117] "RemoveContainer" containerID="6469161b3408f6311e3e475a599e9813687634849322da8ff5e632921bba612a" Jul 06 20:01:19.791221 ip-10-0-130-172 kubenswrapper[2581]: E0706 20:01:19.791201 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6469161b3408f6311e3e475a599e9813687634849322da8ff5e632921bba612a\": container with ID starting with 6469161b3408f6311e3e475a599e9813687634849322da8ff5e632921bba612a not found: ID does not exist" containerID="6469161b3408f6311e3e475a599e9813687634849322da8ff5e632921bba612a" Jul 06 20:01:19.791277 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:19.791229 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6469161b3408f6311e3e475a599e9813687634849322da8ff5e632921bba612a"} err="failed to get container status \"6469161b3408f6311e3e475a599e9813687634849322da8ff5e632921bba612a\": rpc error: code = NotFound desc = could not find container \"6469161b3408f6311e3e475a599e9813687634849322da8ff5e632921bba612a\": container with ID starting with 6469161b3408f6311e3e475a599e9813687634849322da8ff5e632921bba612a not found: ID does not exist" Jul 06 20:01:19.808051 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:19.808018 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7588cc9ccd-7m9ws"] Jul 06 20:01:19.814002 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:19.813977 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7588cc9ccd-7m9ws"] Jul 06 20:01:20.623418 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:20.623386 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7588cc9ccd-fm8tf"] Jul 06 20:01:20.623789 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:20.623775 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12c5da94-35d7-4262-8cf1-6cb83ba4b60f" containerName="cleanup" Jul 06 20:01:20.623848 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:20.623792 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="12c5da94-35d7-4262-8cf1-6cb83ba4b60f" containerName="cleanup" Jul 06 20:01:20.623848 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:20.623821 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33bae915-3dbf-4a90-8d7b-f7980a8b7e59" containerName="authorino" Jul 06 20:01:20.623848 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:20.623827 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="33bae915-3dbf-4a90-8d7b-f7980a8b7e59" containerName="authorino" Jul 06 20:01:20.623848 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:20.623844 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2dbcb90-1a1e-4d77-89f6-e408cbba8728" containerName="manager" Jul 06 20:01:20.624014 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:20.623851 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2dbcb90-1a1e-4d77-89f6-e408cbba8728" containerName="manager" Jul 06 20:01:20.624014 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:20.623914 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="33bae915-3dbf-4a90-8d7b-f7980a8b7e59" containerName="authorino" Jul 06 20:01:20.624014 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:20.623924 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="12c5da94-35d7-4262-8cf1-6cb83ba4b60f" containerName="cleanup" Jul 06 20:01:20.624014 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:20.623931 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2dbcb90-1a1e-4d77-89f6-e408cbba8728" containerName="manager" Jul 06 20:01:20.627359 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:20.627339 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7588cc9ccd-fm8tf" Jul 06 20:01:20.630122 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:20.630097 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-sx4sv\"" Jul 06 20:01:20.630236 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:20.630170 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 20:01:20.630236 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:20.630170 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 20:01:20.637185 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:20.637160 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7588cc9ccd-fm8tf"] Jul 06 20:01:20.719180 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:20.719142 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mqfl\" (UniqueName: \"kubernetes.io/projected/0111e398-087a-4d2a-8a38-e08a03c406f6-kube-api-access-7mqfl\") pod \"maas-controller-7588cc9ccd-fm8tf\" (UID: \"0111e398-087a-4d2a-8a38-e08a03c406f6\") " pod="opendatahub/maas-controller-7588cc9ccd-fm8tf" Jul 06 20:01:20.719397 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:20.719254 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0111e398-087a-4d2a-8a38-e08a03c406f6-webhook-cert\") pod \"maas-controller-7588cc9ccd-fm8tf\" (UID: \"0111e398-087a-4d2a-8a38-e08a03c406f6\") " pod="opendatahub/maas-controller-7588cc9ccd-fm8tf" Jul 06 20:01:20.820826 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:20.820792 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0111e398-087a-4d2a-8a38-e08a03c406f6-webhook-cert\") pod \"maas-controller-7588cc9ccd-fm8tf\" (UID: \"0111e398-087a-4d2a-8a38-e08a03c406f6\") " pod="opendatahub/maas-controller-7588cc9ccd-fm8tf" Jul 06 20:01:20.821196 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:20.820957 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mqfl\" (UniqueName: \"kubernetes.io/projected/0111e398-087a-4d2a-8a38-e08a03c406f6-kube-api-access-7mqfl\") pod \"maas-controller-7588cc9ccd-fm8tf\" (UID: \"0111e398-087a-4d2a-8a38-e08a03c406f6\") " pod="opendatahub/maas-controller-7588cc9ccd-fm8tf" Jul 06 20:01:20.823269 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:20.823237 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0111e398-087a-4d2a-8a38-e08a03c406f6-webhook-cert\") pod \"maas-controller-7588cc9ccd-fm8tf\" (UID: \"0111e398-087a-4d2a-8a38-e08a03c406f6\") " pod="opendatahub/maas-controller-7588cc9ccd-fm8tf" Jul 06 20:01:20.829852 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:20.829820 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mqfl\" (UniqueName: \"kubernetes.io/projected/0111e398-087a-4d2a-8a38-e08a03c406f6-kube-api-access-7mqfl\") pod \"maas-controller-7588cc9ccd-fm8tf\" (UID: \"0111e398-087a-4d2a-8a38-e08a03c406f6\") " pod="opendatahub/maas-controller-7588cc9ccd-fm8tf" Jul 06 20:01:20.938297 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:20.938204 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7588cc9ccd-fm8tf" Jul 06 20:01:21.072799 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:21.072775 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7588cc9ccd-fm8tf"] Jul 06 20:01:21.075063 ip-10-0-130-172 kubenswrapper[2581]: W0706 20:01:21.075035 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0111e398_087a_4d2a_8a38_e08a03c406f6.slice/crio-07adcf3d5e852e107b8cbe8207ae2e604f2288b331a1baff7a981746840744e3 WatchSource:0}: Error finding container 07adcf3d5e852e107b8cbe8207ae2e604f2288b331a1baff7a981746840744e3: Status 404 returned error can't find the container with id 07adcf3d5e852e107b8cbe8207ae2e604f2288b331a1baff7a981746840744e3 Jul 06 20:01:21.095723 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:21.095694 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2dbcb90-1a1e-4d77-89f6-e408cbba8728" path="/var/lib/kubelet/pods/c2dbcb90-1a1e-4d77-89f6-e408cbba8728/volumes" Jul 06 20:01:21.793343 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:21.793281 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7588cc9ccd-fm8tf" event={"ID":"0111e398-087a-4d2a-8a38-e08a03c406f6","Type":"ContainerStarted","Data":"07adcf3d5e852e107b8cbe8207ae2e604f2288b331a1baff7a981746840744e3"} Jul 06 20:01:22.365307 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:22.365281 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 20:01:22.799775 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:22.799738 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7588cc9ccd-fm8tf" event={"ID":"0111e398-087a-4d2a-8a38-e08a03c406f6","Type":"ContainerStarted","Data":"ac14389e2adcede620933f7961da12f91ea35109fef66e807748cbaa9e22d49c"} Jul 06 20:01:22.799969 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:22.799876 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7588cc9ccd-fm8tf" Jul 06 20:01:22.819392 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:22.819313 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7588cc9ccd-fm8tf" podStartSLOduration=1.5335886570000001 podStartE2EDuration="2.819297416s" totalImagesPullingTime="1.285708759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:01:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:01:22.81889968 +0000 UTC m=+782.266136399" watchObservedRunningTime="2026-07-06 20:01:22.819297416 +0000 UTC m=+782.266534133" Jul 06 20:01:33.812728 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:01:33.812688 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7588cc9ccd-fm8tf" Jul 06 20:03:21.052225 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:03:21.052156 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/2.log" Jul 06 20:03:21.052225 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:03:21.052197 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/2.log" Jul 06 20:07:11.629226 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:07:11.629190 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phmfd"] Jul 06 20:07:11.632874 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:07:11.632853 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phmfd" Jul 06 20:07:11.635345 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:07:11.635296 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 20:07:11.635450 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:07:11.635345 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 20:07:11.635450 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:07:11.635360 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tckhr\"" Jul 06 20:07:11.651115 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:07:11.651089 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phmfd"] Jul 06 20:07:11.718383 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:07:11.718312 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c08eece9-6a10-4b09-a4c3-54389d993c67-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-phmfd\" (UID: \"c08eece9-6a10-4b09-a4c3-54389d993c67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phmfd" Jul 06 20:07:11.718567 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:07:11.718519 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nfvp\" (UniqueName: \"kubernetes.io/projected/c08eece9-6a10-4b09-a4c3-54389d993c67-kube-api-access-6nfvp\") pod \"kuadrant-operator-controller-manager-86878c769f-phmfd\" (UID: \"c08eece9-6a10-4b09-a4c3-54389d993c67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phmfd" Jul 06 20:07:11.819695 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:07:11.819657 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nfvp\" (UniqueName: \"kubernetes.io/projected/c08eece9-6a10-4b09-a4c3-54389d993c67-kube-api-access-6nfvp\") pod \"kuadrant-operator-controller-manager-86878c769f-phmfd\" (UID: \"c08eece9-6a10-4b09-a4c3-54389d993c67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phmfd" Jul 06 20:07:11.819869 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:07:11.819723 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c08eece9-6a10-4b09-a4c3-54389d993c67-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-phmfd\" (UID: \"c08eece9-6a10-4b09-a4c3-54389d993c67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phmfd" Jul 06 20:07:11.820111 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:07:11.820095 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c08eece9-6a10-4b09-a4c3-54389d993c67-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-phmfd\" (UID: \"c08eece9-6a10-4b09-a4c3-54389d993c67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phmfd" Jul 06 20:07:11.830115 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:07:11.830081 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nfvp\" (UniqueName: \"kubernetes.io/projected/c08eece9-6a10-4b09-a4c3-54389d993c67-kube-api-access-6nfvp\") pod \"kuadrant-operator-controller-manager-86878c769f-phmfd\" (UID: \"c08eece9-6a10-4b09-a4c3-54389d993c67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phmfd" Jul 06 20:07:11.943552 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:07:11.943461 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phmfd" Jul 06 20:07:12.086760 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:07:12.086730 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phmfd"] Jul 06 20:07:12.088610 ip-10-0-130-172 kubenswrapper[2581]: W0706 20:07:12.088581 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc08eece9_6a10_4b09_a4c3_54389d993c67.slice/crio-ed455bc5b09f77567834f5c71c1e3a16e11841a4abc86d7563c8a35a4eacbca6 WatchSource:0}: Error finding container ed455bc5b09f77567834f5c71c1e3a16e11841a4abc86d7563c8a35a4eacbca6: Status 404 returned error can't find the container with id ed455bc5b09f77567834f5c71c1e3a16e11841a4abc86d7563c8a35a4eacbca6 Jul 06 20:07:12.546309 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:07:12.546266 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phmfd" event={"ID":"c08eece9-6a10-4b09-a4c3-54389d993c67","Type":"ContainerStarted","Data":"e90b51f01d52e7360c97a0df508294077a652fdac13fde8e11c77df87dc58651"} Jul 06 20:07:12.546549 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:07:12.546351 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phmfd" Jul 06 20:07:12.546549 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:07:12.546376 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phmfd" event={"ID":"c08eece9-6a10-4b09-a4c3-54389d993c67","Type":"ContainerStarted","Data":"ed455bc5b09f77567834f5c71c1e3a16e11841a4abc86d7563c8a35a4eacbca6"} Jul 06 20:07:12.571643 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:07:12.571599 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phmfd" podStartSLOduration=1.571582418 podStartE2EDuration="1.571582418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:07:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:07:12.569514317 +0000 UTC m=+1132.016751036" watchObservedRunningTime="2026-07-06 20:07:12.571582418 +0000 UTC m=+1132.018819135" Jul 06 20:07:23.553603 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:07:23.553572 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-phmfd" Jul 06 20:08:21.085296 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:08:21.085194 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/2.log" Jul 06 20:08:21.089302 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:08:21.086918 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/2.log" Jul 06 20:13:21.118764 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:21.118666 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/2.log" Jul 06 20:13:21.123993 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:21.123971 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/2.log" Jul 06 20:13:26.804525 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.804488 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh"] Jul 06 20:13:26.808189 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.808166 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.810586 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.810564 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-81f0cb7a-openshift-default-dockercfg-nw2sf\"" Jul 06 20:13:26.815459 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.815310 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-credential-socket\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.815459 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.815414 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-envoy\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.815459 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.815439 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-workload-socket\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.815664 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.815575 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-data\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.815664 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.815608 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-workload-certs\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.815664 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.815639 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxvr8\" (UniqueName: \"kubernetes.io/projected/0ad35783-0f3e-4e08-b389-0def4632f14f-kube-api-access-vxvr8\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.815824 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.815668 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-podinfo\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.815824 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.815683 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ad35783-0f3e-4e08-b389-0def4632f14f-istiod-ca-cert\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.815824 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.815766 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-token\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.837018 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.836984 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh"] Jul 06 20:13:26.916501 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.916464 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-data\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.916696 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.916506 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-workload-certs\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.916696 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.916525 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxvr8\" (UniqueName: \"kubernetes.io/projected/0ad35783-0f3e-4e08-b389-0def4632f14f-kube-api-access-vxvr8\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.916696 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.916634 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-podinfo\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.916696 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.916668 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ad35783-0f3e-4e08-b389-0def4632f14f-istiod-ca-cert\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.916920 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.916750 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-token\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.916920 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.916798 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-credential-socket\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.916920 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.916826 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-data\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.916920 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.916899 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-envoy\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.916920 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.916907 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-workload-certs\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.917205 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.916925 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-workload-socket\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.917205 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.917129 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-credential-socket\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.917278 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.917221 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-workload-socket\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.917378 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.917358 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ad35783-0f3e-4e08-b389-0def4632f14f-istiod-ca-cert\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.918918 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.918898 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-envoy\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.919149 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.919133 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-podinfo\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.927298 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.927272 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-token\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:26.927423 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:26.927363 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxvr8\" (UniqueName: \"kubernetes.io/projected/0ad35783-0f3e-4e08-b389-0def4632f14f-kube-api-access-vxvr8\") pod \"e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:27.118765 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:27.118681 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:27.270029 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:27.270004 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh"] Jul 06 20:13:27.272308 ip-10-0-130-172 kubenswrapper[2581]: W0706 20:13:27.272273 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ad35783_0f3e_4e08_b389_0def4632f14f.slice/crio-69f1af31c52fcb77b2431d956ef380663c26bd61b4407fe26a50da636c234461 WatchSource:0}: Error finding container 69f1af31c52fcb77b2431d956ef380663c26bd61b4407fe26a50da636c234461: Status 404 returned error can't find the container with id 69f1af31c52fcb77b2431d956ef380663c26bd61b4407fe26a50da636c234461 Jul 06 20:13:27.274639 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:27.274610 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:13:27.274708 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:27.274675 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:13:27.274746 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:27.274706 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:13:27.461100 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:27.461000 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" event={"ID":"0ad35783-0f3e-4e08-b389-0def4632f14f","Type":"ContainerStarted","Data":"71a018df3f11d864ef44ad7dfee9442ba84dba68f073bf8a119620a85f0db7e4"} Jul 06 20:13:27.461100 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:27.461071 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" event={"ID":"0ad35783-0f3e-4e08-b389-0def4632f14f","Type":"ContainerStarted","Data":"69f1af31c52fcb77b2431d956ef380663c26bd61b4407fe26a50da636c234461"} Jul 06 20:13:27.488397 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:27.488349 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" podStartSLOduration=1.488308124 podStartE2EDuration="1.488308124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:13:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:13:27.487300513 +0000 UTC m=+1506.934537232" watchObservedRunningTime="2026-07-06 20:13:27.488308124 +0000 UTC m=+1506.935544843" Jul 06 20:13:28.119194 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:28.119159 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:28.123880 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:28.123857 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:28.466806 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:28.466721 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:28.467707 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:28.467686 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:33.251407 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:33.251367 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh"] Jul 06 20:13:33.252191 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:33.252137 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" podUID="0ad35783-0f3e-4e08-b389-0def4632f14f" containerName="istio-proxy" containerID="cri-o://71a018df3f11d864ef44ad7dfee9442ba84dba68f073bf8a119620a85f0db7e4" gracePeriod=30 Jul 06 20:13:39.118143 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.118108 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7"] Jul 06 20:13:39.124181 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.124159 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.127207 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.127182 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-663315aa-openshift-default-dockercfg-6dnzt\"" Jul 06 20:13:39.135908 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.135879 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7"] Jul 06 20:13:39.236284 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.236242 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-token\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.236496 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.236408 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istiod-ca-cert\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.236496 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.236461 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-workload-certs\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.236589 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.236493 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz7tt\" (UniqueName: \"kubernetes.io/projected/75a41f7c-c45f-45ce-aa7d-af7064c6f558-kube-api-access-mz7tt\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.236627 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.236587 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-credential-socket\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.236677 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.236641 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-envoy\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.236722 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.236708 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-workload-socket\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.236766 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.236739 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-data\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.236816 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.236795 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-podinfo\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.337678 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.337641 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-token\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.337870 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.337705 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istiod-ca-cert\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.337870 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.337743 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-workload-certs\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.337870 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.337770 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz7tt\" (UniqueName: \"kubernetes.io/projected/75a41f7c-c45f-45ce-aa7d-af7064c6f558-kube-api-access-mz7tt\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.337870 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.337812 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-credential-socket\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.337870 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.337863 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-envoy\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.338143 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.337914 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-workload-socket\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.338143 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.337953 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-data\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.338143 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.337986 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-podinfo\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.338300 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.338254 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-workload-certs\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.338390 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.338345 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-credential-socket\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.338455 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.338407 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-workload-socket\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.338518 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.338461 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-data\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.338696 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.338664 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istiod-ca-cert\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.340315 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.340284 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-envoy\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.340442 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.340384 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-podinfo\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.345809 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.345783 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz7tt\" (UniqueName: \"kubernetes.io/projected/75a41f7c-c45f-45ce-aa7d-af7064c6f558-kube-api-access-mz7tt\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.346274 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.346248 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-token\") pod \"e2e-ait-663315aa-openshift-default-8468596875-jksw7\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.437068 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.437033 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:39.585890 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.585859 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7"] Jul 06 20:13:39.590458 ip-10-0-130-172 kubenswrapper[2581]: W0706 20:13:39.590431 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75a41f7c_c45f_45ce_aa7d_af7064c6f558.slice/crio-e2a9b3a87e2b01dc7705d0ee1d71d39be775b7c4e5b2bddc55a2bd93d95d002d WatchSource:0}: Error finding container e2a9b3a87e2b01dc7705d0ee1d71d39be775b7c4e5b2bddc55a2bd93d95d002d: Status 404 returned error can't find the container with id e2a9b3a87e2b01dc7705d0ee1d71d39be775b7c4e5b2bddc55a2bd93d95d002d Jul 06 20:13:39.592699 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.592666 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:13:39.592787 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.592735 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:13:39.592787 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.592764 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:13:39.700418 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.700386 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:39.743561 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.743526 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-data\") pod \"0ad35783-0f3e-4e08-b389-0def4632f14f\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " Jul 06 20:13:39.743738 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.743574 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-podinfo\") pod \"0ad35783-0f3e-4e08-b389-0def4632f14f\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " Jul 06 20:13:39.743738 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.743614 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-credential-socket\") pod \"0ad35783-0f3e-4e08-b389-0def4632f14f\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " Jul 06 20:13:39.743738 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.743639 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ad35783-0f3e-4e08-b389-0def4632f14f-istiod-ca-cert\") pod \"0ad35783-0f3e-4e08-b389-0def4632f14f\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " Jul 06 20:13:39.743916 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.743742 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-token\") pod \"0ad35783-0f3e-4e08-b389-0def4632f14f\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " Jul 06 20:13:39.743916 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.743802 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-workload-certs\") pod \"0ad35783-0f3e-4e08-b389-0def4632f14f\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " Jul 06 20:13:39.743916 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.743833 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-workload-socket\") pod \"0ad35783-0f3e-4e08-b389-0def4632f14f\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " Jul 06 20:13:39.743916 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.743862 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxvr8\" (UniqueName: \"kubernetes.io/projected/0ad35783-0f3e-4e08-b389-0def4632f14f-kube-api-access-vxvr8\") pod \"0ad35783-0f3e-4e08-b389-0def4632f14f\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " Jul 06 20:13:39.743916 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.743905 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-envoy\") pod \"0ad35783-0f3e-4e08-b389-0def4632f14f\" (UID: \"0ad35783-0f3e-4e08-b389-0def4632f14f\") " Jul 06 20:13:39.744294 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.744271 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-data" (OuterVolumeSpecName: "istio-data") pod "0ad35783-0f3e-4e08-b389-0def4632f14f" (UID: "0ad35783-0f3e-4e08-b389-0def4632f14f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:13:39.745130 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.744769 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0ad35783-0f3e-4e08-b389-0def4632f14f" (UID: "0ad35783-0f3e-4e08-b389-0def4632f14f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:13:39.745130 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.744796 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0ad35783-0f3e-4e08-b389-0def4632f14f" (UID: "0ad35783-0f3e-4e08-b389-0def4632f14f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:13:39.745130 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.744994 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0ad35783-0f3e-4e08-b389-0def4632f14f" (UID: "0ad35783-0f3e-4e08-b389-0def4632f14f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:13:39.745130 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.745115 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ad35783-0f3e-4e08-b389-0def4632f14f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0ad35783-0f3e-4e08-b389-0def4632f14f" (UID: "0ad35783-0f3e-4e08-b389-0def4632f14f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:13:39.747204 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.747180 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-token" (OuterVolumeSpecName: "istio-token") pod "0ad35783-0f3e-4e08-b389-0def4632f14f" (UID: "0ad35783-0f3e-4e08-b389-0def4632f14f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:13:39.747305 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.747233 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0ad35783-0f3e-4e08-b389-0def4632f14f" (UID: "0ad35783-0f3e-4e08-b389-0def4632f14f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:13:39.747534 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.747510 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0ad35783-0f3e-4e08-b389-0def4632f14f" (UID: "0ad35783-0f3e-4e08-b389-0def4632f14f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:13:39.747729 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.747705 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ad35783-0f3e-4e08-b389-0def4632f14f-kube-api-access-vxvr8" (OuterVolumeSpecName: "kube-api-access-vxvr8") pod "0ad35783-0f3e-4e08-b389-0def4632f14f" (UID: "0ad35783-0f3e-4e08-b389-0def4632f14f"). InnerVolumeSpecName "kube-api-access-vxvr8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:13:39.845632 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.845597 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-token\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:13:39.845632 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.845631 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-workload-certs\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:13:39.845832 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.845643 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-workload-socket\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:13:39.845832 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.845655 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxvr8\" (UniqueName: \"kubernetes.io/projected/0ad35783-0f3e-4e08-b389-0def4632f14f-kube-api-access-vxvr8\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:13:39.845832 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.845667 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-envoy\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:13:39.845832 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.845680 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-data\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:13:39.845832 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.845690 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ad35783-0f3e-4e08-b389-0def4632f14f-istio-podinfo\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:13:39.845832 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.845701 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad35783-0f3e-4e08-b389-0def4632f14f-credential-socket\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:13:39.845832 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:39.845712 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ad35783-0f3e-4e08-b389-0def4632f14f-istiod-ca-cert\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:13:40.547257 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:40.547222 2581 generic.go:356] "Generic (PLEG): container finished" podID="0ad35783-0f3e-4e08-b389-0def4632f14f" containerID="71a018df3f11d864ef44ad7dfee9442ba84dba68f073bf8a119620a85f0db7e4" exitCode=0 Jul 06 20:13:40.547807 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:40.547277 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" Jul 06 20:13:40.547807 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:40.547330 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" event={"ID":"0ad35783-0f3e-4e08-b389-0def4632f14f","Type":"ContainerDied","Data":"71a018df3f11d864ef44ad7dfee9442ba84dba68f073bf8a119620a85f0db7e4"} Jul 06 20:13:40.547807 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:40.547411 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh" event={"ID":"0ad35783-0f3e-4e08-b389-0def4632f14f","Type":"ContainerDied","Data":"69f1af31c52fcb77b2431d956ef380663c26bd61b4407fe26a50da636c234461"} Jul 06 20:13:40.547807 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:40.547442 2581 scope.go:117] "RemoveContainer" containerID="71a018df3f11d864ef44ad7dfee9442ba84dba68f073bf8a119620a85f0db7e4" Jul 06 20:13:40.549467 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:40.549430 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" event={"ID":"75a41f7c-c45f-45ce-aa7d-af7064c6f558","Type":"ContainerStarted","Data":"91b40a6b37835a7ac60588ab386a1b0d226b84906fa64b3387add317365c3d96"} Jul 06 20:13:40.549573 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:40.549484 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" event={"ID":"75a41f7c-c45f-45ce-aa7d-af7064c6f558","Type":"ContainerStarted","Data":"e2a9b3a87e2b01dc7705d0ee1d71d39be775b7c4e5b2bddc55a2bd93d95d002d"} Jul 06 20:13:40.558304 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:40.558290 2581 scope.go:117] "RemoveContainer" containerID="71a018df3f11d864ef44ad7dfee9442ba84dba68f073bf8a119620a85f0db7e4" Jul 06 20:13:40.558557 ip-10-0-130-172 kubenswrapper[2581]: E0706 20:13:40.558541 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71a018df3f11d864ef44ad7dfee9442ba84dba68f073bf8a119620a85f0db7e4\": container with ID starting with 71a018df3f11d864ef44ad7dfee9442ba84dba68f073bf8a119620a85f0db7e4 not found: ID does not exist" containerID="71a018df3f11d864ef44ad7dfee9442ba84dba68f073bf8a119620a85f0db7e4" Jul 06 20:13:40.558613 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:40.558564 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71a018df3f11d864ef44ad7dfee9442ba84dba68f073bf8a119620a85f0db7e4"} err="failed to get container status \"71a018df3f11d864ef44ad7dfee9442ba84dba68f073bf8a119620a85f0db7e4\": rpc error: code = NotFound desc = could not find container \"71a018df3f11d864ef44ad7dfee9442ba84dba68f073bf8a119620a85f0db7e4\": container with ID starting with 71a018df3f11d864ef44ad7dfee9442ba84dba68f073bf8a119620a85f0db7e4 not found: ID does not exist" Jul 06 20:13:40.575242 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:40.575189 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" podStartSLOduration=1.5751754519999999 podStartE2EDuration="1.575175452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:13:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:13:40.574141991 +0000 UTC m=+1520.021378710" watchObservedRunningTime="2026-07-06 20:13:40.575175452 +0000 UTC m=+1520.022412169" Jul 06 20:13:40.593724 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:40.593690 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh"] Jul 06 20:13:40.599853 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:40.599780 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-81f0cb7a-openshift-default-9675b68ff-tptwh"] Jul 06 20:13:41.096038 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:41.096008 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ad35783-0f3e-4e08-b389-0def4632f14f" path="/var/lib/kubelet/pods/0ad35783-0f3e-4e08-b389-0def4632f14f/volumes" Jul 06 20:13:41.437587 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:41.437465 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:41.443056 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:41.443017 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:41.556202 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:41.556171 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:41.557552 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:41.557530 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:51.467054 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:51.467018 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7"] Jul 06 20:13:51.467512 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:51.467222 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" podUID="75a41f7c-c45f-45ce-aa7d-af7064c6f558" containerName="istio-proxy" containerID="cri-o://91b40a6b37835a7ac60588ab386a1b0d226b84906fa64b3387add317365c3d96" gracePeriod=30 Jul 06 20:13:51.924761 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:51.924713 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:52.064315 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.064282 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-podinfo\") pod \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " Jul 06 20:13:52.064530 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.064359 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-workload-certs\") pod \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " Jul 06 20:13:52.064530 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.064404 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-workload-socket\") pod \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " Jul 06 20:13:52.064530 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.064430 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mz7tt\" (UniqueName: \"kubernetes.io/projected/75a41f7c-c45f-45ce-aa7d-af7064c6f558-kube-api-access-mz7tt\") pod \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " Jul 06 20:13:52.064530 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.064464 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-envoy\") pod \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " Jul 06 20:13:52.064530 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.064499 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-data\") pod \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " Jul 06 20:13:52.064796 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.064589 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-credential-socket\") pod \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " Jul 06 20:13:52.064796 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.064642 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istiod-ca-cert\") pod \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " Jul 06 20:13:52.064796 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.064687 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-token\") pod \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\" (UID: \"75a41f7c-c45f-45ce-aa7d-af7064c6f558\") " Jul 06 20:13:52.064796 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.064710 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "75a41f7c-c45f-45ce-aa7d-af7064c6f558" (UID: "75a41f7c-c45f-45ce-aa7d-af7064c6f558"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:13:52.064796 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.064723 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "75a41f7c-c45f-45ce-aa7d-af7064c6f558" (UID: "75a41f7c-c45f-45ce-aa7d-af7064c6f558"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:13:52.065058 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.064904 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "75a41f7c-c45f-45ce-aa7d-af7064c6f558" (UID: "75a41f7c-c45f-45ce-aa7d-af7064c6f558"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:13:52.065058 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.065015 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-credential-socket\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:13:52.065058 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.065029 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-workload-certs\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:13:52.065058 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.065030 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "75a41f7c-c45f-45ce-aa7d-af7064c6f558" (UID: "75a41f7c-c45f-45ce-aa7d-af7064c6f558"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:13:52.065058 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.065037 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-workload-socket\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:13:52.065367 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.065343 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-data" (OuterVolumeSpecName: "istio-data") pod "75a41f7c-c45f-45ce-aa7d-af7064c6f558" (UID: "75a41f7c-c45f-45ce-aa7d-af7064c6f558"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:13:52.066955 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.066934 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "75a41f7c-c45f-45ce-aa7d-af7064c6f558" (UID: "75a41f7c-c45f-45ce-aa7d-af7064c6f558"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:13:52.066955 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.066941 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75a41f7c-c45f-45ce-aa7d-af7064c6f558-kube-api-access-mz7tt" (OuterVolumeSpecName: "kube-api-access-mz7tt") pod "75a41f7c-c45f-45ce-aa7d-af7064c6f558" (UID: "75a41f7c-c45f-45ce-aa7d-af7064c6f558"). InnerVolumeSpecName "kube-api-access-mz7tt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:13:52.067366 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.067342 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "75a41f7c-c45f-45ce-aa7d-af7064c6f558" (UID: "75a41f7c-c45f-45ce-aa7d-af7064c6f558"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:13:52.067448 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.067349 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-token" (OuterVolumeSpecName: "istio-token") pod "75a41f7c-c45f-45ce-aa7d-af7064c6f558" (UID: "75a41f7c-c45f-45ce-aa7d-af7064c6f558"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:13:52.165983 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.165952 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-podinfo\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:13:52.165983 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.165980 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mz7tt\" (UniqueName: \"kubernetes.io/projected/75a41f7c-c45f-45ce-aa7d-af7064c6f558-kube-api-access-mz7tt\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:13:52.165983 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.165991 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-envoy\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:13:52.166239 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.165999 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-data\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:13:52.166239 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.166007 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istiod-ca-cert\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:13:52.166239 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.166014 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a41f7c-c45f-45ce-aa7d-af7064c6f558-istio-token\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:13:52.618655 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.618617 2581 generic.go:356] "Generic (PLEG): container finished" podID="75a41f7c-c45f-45ce-aa7d-af7064c6f558" containerID="91b40a6b37835a7ac60588ab386a1b0d226b84906fa64b3387add317365c3d96" exitCode=0 Jul 06 20:13:52.619062 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.618662 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" Jul 06 20:13:52.619062 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.618692 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" event={"ID":"75a41f7c-c45f-45ce-aa7d-af7064c6f558","Type":"ContainerDied","Data":"91b40a6b37835a7ac60588ab386a1b0d226b84906fa64b3387add317365c3d96"} Jul 06 20:13:52.619062 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.618735 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7" event={"ID":"75a41f7c-c45f-45ce-aa7d-af7064c6f558","Type":"ContainerDied","Data":"e2a9b3a87e2b01dc7705d0ee1d71d39be775b7c4e5b2bddc55a2bd93d95d002d"} Jul 06 20:13:52.619062 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.618760 2581 scope.go:117] "RemoveContainer" containerID="91b40a6b37835a7ac60588ab386a1b0d226b84906fa64b3387add317365c3d96" Jul 06 20:13:52.628254 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.628235 2581 scope.go:117] "RemoveContainer" containerID="91b40a6b37835a7ac60588ab386a1b0d226b84906fa64b3387add317365c3d96" Jul 06 20:13:52.628528 ip-10-0-130-172 kubenswrapper[2581]: E0706 20:13:52.628509 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91b40a6b37835a7ac60588ab386a1b0d226b84906fa64b3387add317365c3d96\": container with ID starting with 91b40a6b37835a7ac60588ab386a1b0d226b84906fa64b3387add317365c3d96 not found: ID does not exist" containerID="91b40a6b37835a7ac60588ab386a1b0d226b84906fa64b3387add317365c3d96" Jul 06 20:13:52.628583 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.628536 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91b40a6b37835a7ac60588ab386a1b0d226b84906fa64b3387add317365c3d96"} err="failed to get container status \"91b40a6b37835a7ac60588ab386a1b0d226b84906fa64b3387add317365c3d96\": rpc error: code = NotFound desc = could not find container \"91b40a6b37835a7ac60588ab386a1b0d226b84906fa64b3387add317365c3d96\": container with ID starting with 91b40a6b37835a7ac60588ab386a1b0d226b84906fa64b3387add317365c3d96 not found: ID does not exist" Jul 06 20:13:52.641233 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.641207 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7"] Jul 06 20:13:52.645774 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:52.645751 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-663315aa-openshift-default-8468596875-jksw7"] Jul 06 20:13:53.096168 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:53.096126 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75a41f7c-c45f-45ce-aa7d-af7064c6f558" path="/var/lib/kubelet/pods/75a41f7c-c45f-45ce-aa7d-af7064c6f558/volumes" Jul 06 20:13:57.383952 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.383914 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f"] Jul 06 20:13:57.384485 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.384469 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ad35783-0f3e-4e08-b389-0def4632f14f" containerName="istio-proxy" Jul 06 20:13:57.384485 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.384486 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ad35783-0f3e-4e08-b389-0def4632f14f" containerName="istio-proxy" Jul 06 20:13:57.384587 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.384515 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75a41f7c-c45f-45ce-aa7d-af7064c6f558" containerName="istio-proxy" Jul 06 20:13:57.384587 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.384521 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="75a41f7c-c45f-45ce-aa7d-af7064c6f558" containerName="istio-proxy" Jul 06 20:13:57.384587 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.384585 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ad35783-0f3e-4e08-b389-0def4632f14f" containerName="istio-proxy" Jul 06 20:13:57.384684 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.384592 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="75a41f7c-c45f-45ce-aa7d-af7064c6f558" containerName="istio-proxy" Jul 06 20:13:57.387373 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.387354 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.389882 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.389858 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-2c63cbed-openshift-default-dockercfg-rfv4w\"" Jul 06 20:13:57.400915 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.400889 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f"] Jul 06 20:13:57.516974 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.516942 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-data\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.517137 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.516974 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wzr7\" (UniqueName: \"kubernetes.io/projected/85caa676-1e13-4197-835f-eaaffb8d7d11-kube-api-access-5wzr7\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.517137 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.517010 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-credential-socket\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.517137 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.517061 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-token\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.517137 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.517094 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-envoy\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.517137 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.517119 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85caa676-1e13-4197-835f-eaaffb8d7d11-istiod-ca-cert\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.517305 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.517146 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-workload-certs\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.517305 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.517195 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-workload-socket\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.517305 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.517247 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-podinfo\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.618616 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.618580 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-envoy\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.618616 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.618623 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85caa676-1e13-4197-835f-eaaffb8d7d11-istiod-ca-cert\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.618825 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.618787 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-workload-certs\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.618825 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.618817 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-workload-socket\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.618937 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.618847 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-podinfo\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.618937 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.618910 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-data\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.618937 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.618926 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wzr7\" (UniqueName: \"kubernetes.io/projected/85caa676-1e13-4197-835f-eaaffb8d7d11-kube-api-access-5wzr7\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.619093 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.618953 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-credential-socket\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.619093 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.618996 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-token\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.619205 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.619135 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-workload-certs\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.619205 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.619149 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-workload-socket\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.619356 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.619274 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85caa676-1e13-4197-835f-eaaffb8d7d11-istiod-ca-cert\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.619470 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.619449 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-data\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.619534 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.619512 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-credential-socket\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.620986 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.620966 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-envoy\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.621288 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.621269 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-podinfo\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.626494 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.626472 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-token\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.626605 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.626512 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wzr7\" (UniqueName: \"kubernetes.io/projected/85caa676-1e13-4197-835f-eaaffb8d7d11-kube-api-access-5wzr7\") pod \"e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.699755 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.699668 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:57.838558 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.838534 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f"] Jul 06 20:13:57.840270 ip-10-0-130-172 kubenswrapper[2581]: W0706 20:13:57.840239 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85caa676_1e13_4197_835f_eaaffb8d7d11.slice/crio-2e5b58f0d73d5cbd066798c65b3f52d99be5d5f019420c23ccb31b8a55d4388d WatchSource:0}: Error finding container 2e5b58f0d73d5cbd066798c65b3f52d99be5d5f019420c23ccb31b8a55d4388d: Status 404 returned error can't find the container with id 2e5b58f0d73d5cbd066798c65b3f52d99be5d5f019420c23ccb31b8a55d4388d Jul 06 20:13:57.842973 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.842935 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:13:57.843105 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.843016 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:13:57.843105 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:57.843062 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:13:58.653039 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:58.653006 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" event={"ID":"85caa676-1e13-4197-835f-eaaffb8d7d11","Type":"ContainerStarted","Data":"332303accac61368dd97093b85e8537546e3eb77bd456c1b6c2542e74a189383"} Jul 06 20:13:58.653435 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:58.653058 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" event={"ID":"85caa676-1e13-4197-835f-eaaffb8d7d11","Type":"ContainerStarted","Data":"2e5b58f0d73d5cbd066798c65b3f52d99be5d5f019420c23ccb31b8a55d4388d"} Jul 06 20:13:58.677965 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:58.677916 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" podStartSLOduration=1.6779016690000002 podStartE2EDuration="1.677901669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:13:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:13:58.675293725 +0000 UTC m=+1538.122530442" watchObservedRunningTime="2026-07-06 20:13:58.677901669 +0000 UTC m=+1538.125138387" Jul 06 20:13:58.700464 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:58.700435 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:58.705298 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:58.705276 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:59.658822 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:59.658797 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:13:59.659852 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:13:59.659831 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:14:03.181603 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:03.181566 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f"] Jul 06 20:14:03.182373 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:03.182275 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" podUID="85caa676-1e13-4197-835f-eaaffb8d7d11" containerName="istio-proxy" containerID="cri-o://332303accac61368dd97093b85e8537546e3eb77bd456c1b6c2542e74a189383" gracePeriod=30 Jul 06 20:14:08.230114 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.230091 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:14:08.243942 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.243917 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wzr7\" (UniqueName: \"kubernetes.io/projected/85caa676-1e13-4197-835f-eaaffb8d7d11-kube-api-access-5wzr7\") pod \"85caa676-1e13-4197-835f-eaaffb8d7d11\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " Jul 06 20:14:08.244017 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.243964 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-workload-socket\") pod \"85caa676-1e13-4197-835f-eaaffb8d7d11\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " Jul 06 20:14:08.244017 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.244003 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-workload-certs\") pod \"85caa676-1e13-4197-835f-eaaffb8d7d11\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " Jul 06 20:14:08.244101 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.244045 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-podinfo\") pod \"85caa676-1e13-4197-835f-eaaffb8d7d11\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " Jul 06 20:14:08.244151 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.244105 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-token\") pod \"85caa676-1e13-4197-835f-eaaffb8d7d11\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " Jul 06 20:14:08.244151 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.244140 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-credential-socket\") pod \"85caa676-1e13-4197-835f-eaaffb8d7d11\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " Jul 06 20:14:08.244255 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.244188 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-envoy\") pod \"85caa676-1e13-4197-835f-eaaffb8d7d11\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " Jul 06 20:14:08.244255 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.244218 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-data\") pod \"85caa676-1e13-4197-835f-eaaffb8d7d11\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " Jul 06 20:14:08.244388 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.244287 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85caa676-1e13-4197-835f-eaaffb8d7d11-istiod-ca-cert\") pod \"85caa676-1e13-4197-835f-eaaffb8d7d11\" (UID: \"85caa676-1e13-4197-835f-eaaffb8d7d11\") " Jul 06 20:14:08.244388 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.244298 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "85caa676-1e13-4197-835f-eaaffb8d7d11" (UID: "85caa676-1e13-4197-835f-eaaffb8d7d11"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:14:08.244388 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.244312 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "85caa676-1e13-4197-835f-eaaffb8d7d11" (UID: "85caa676-1e13-4197-835f-eaaffb8d7d11"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:14:08.244542 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.244515 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "85caa676-1e13-4197-835f-eaaffb8d7d11" (UID: "85caa676-1e13-4197-835f-eaaffb8d7d11"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:14:08.244711 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.244667 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-workload-socket\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:14:08.244891 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.244836 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-workload-certs\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:14:08.244891 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.244863 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-credential-socket\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:14:08.244891 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.244833 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85caa676-1e13-4197-835f-eaaffb8d7d11-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "85caa676-1e13-4197-835f-eaaffb8d7d11" (UID: "85caa676-1e13-4197-835f-eaaffb8d7d11"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:14:08.245424 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.245394 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-data" (OuterVolumeSpecName: "istio-data") pod "85caa676-1e13-4197-835f-eaaffb8d7d11" (UID: "85caa676-1e13-4197-835f-eaaffb8d7d11"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:14:08.246434 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.246385 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85caa676-1e13-4197-835f-eaaffb8d7d11-kube-api-access-5wzr7" (OuterVolumeSpecName: "kube-api-access-5wzr7") pod "85caa676-1e13-4197-835f-eaaffb8d7d11" (UID: "85caa676-1e13-4197-835f-eaaffb8d7d11"). InnerVolumeSpecName "kube-api-access-5wzr7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:14:08.246854 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.246824 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "85caa676-1e13-4197-835f-eaaffb8d7d11" (UID: "85caa676-1e13-4197-835f-eaaffb8d7d11"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:14:08.246927 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.246883 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-token" (OuterVolumeSpecName: "istio-token") pod "85caa676-1e13-4197-835f-eaaffb8d7d11" (UID: "85caa676-1e13-4197-835f-eaaffb8d7d11"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:14:08.247097 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.247083 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "85caa676-1e13-4197-835f-eaaffb8d7d11" (UID: "85caa676-1e13-4197-835f-eaaffb8d7d11"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:14:08.346278 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.346238 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wzr7\" (UniqueName: \"kubernetes.io/projected/85caa676-1e13-4197-835f-eaaffb8d7d11-kube-api-access-5wzr7\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:14:08.346278 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.346276 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-podinfo\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:14:08.346499 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.346290 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-token\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:14:08.346499 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.346302 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-envoy\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:14:08.346499 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.346314 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/85caa676-1e13-4197-835f-eaaffb8d7d11-istio-data\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:14:08.346499 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.346355 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/85caa676-1e13-4197-835f-eaaffb8d7d11-istiod-ca-cert\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:14:08.708710 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.708679 2581 generic.go:356] "Generic (PLEG): container finished" podID="85caa676-1e13-4197-835f-eaaffb8d7d11" containerID="332303accac61368dd97093b85e8537546e3eb77bd456c1b6c2542e74a189383" exitCode=0 Jul 06 20:14:08.708896 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.708727 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" Jul 06 20:14:08.708896 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.708770 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" event={"ID":"85caa676-1e13-4197-835f-eaaffb8d7d11","Type":"ContainerDied","Data":"332303accac61368dd97093b85e8537546e3eb77bd456c1b6c2542e74a189383"} Jul 06 20:14:08.708896 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.708824 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f" event={"ID":"85caa676-1e13-4197-835f-eaaffb8d7d11","Type":"ContainerDied","Data":"2e5b58f0d73d5cbd066798c65b3f52d99be5d5f019420c23ccb31b8a55d4388d"} Jul 06 20:14:08.708896 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.708849 2581 scope.go:117] "RemoveContainer" containerID="332303accac61368dd97093b85e8537546e3eb77bd456c1b6c2542e74a189383" Jul 06 20:14:08.718840 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.718821 2581 scope.go:117] "RemoveContainer" containerID="332303accac61368dd97093b85e8537546e3eb77bd456c1b6c2542e74a189383" Jul 06 20:14:08.719081 ip-10-0-130-172 kubenswrapper[2581]: E0706 20:14:08.719062 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"332303accac61368dd97093b85e8537546e3eb77bd456c1b6c2542e74a189383\": container with ID starting with 332303accac61368dd97093b85e8537546e3eb77bd456c1b6c2542e74a189383 not found: ID does not exist" containerID="332303accac61368dd97093b85e8537546e3eb77bd456c1b6c2542e74a189383" Jul 06 20:14:08.719126 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.719089 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"332303accac61368dd97093b85e8537546e3eb77bd456c1b6c2542e74a189383"} err="failed to get container status \"332303accac61368dd97093b85e8537546e3eb77bd456c1b6c2542e74a189383\": rpc error: code = NotFound desc = could not find container \"332303accac61368dd97093b85e8537546e3eb77bd456c1b6c2542e74a189383\": container with ID starting with 332303accac61368dd97093b85e8537546e3eb77bd456c1b6c2542e74a189383 not found: ID does not exist" Jul 06 20:14:08.733584 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.733562 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f"] Jul 06 20:14:08.738033 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:08.738012 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-2c63cbed-openshift-default-6f74dd8d59-pbg9f"] Jul 06 20:14:09.095306 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:14:09.095276 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85caa676-1e13-4197-835f-eaaffb8d7d11" path="/var/lib/kubelet/pods/85caa676-1e13-4197-835f-eaaffb8d7d11/volumes" Jul 06 20:15:43.878222 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:43.878190 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr"] Jul 06 20:15:43.878697 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:43.878683 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85caa676-1e13-4197-835f-eaaffb8d7d11" containerName="istio-proxy" Jul 06 20:15:43.878753 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:43.878699 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="85caa676-1e13-4197-835f-eaaffb8d7d11" containerName="istio-proxy" Jul 06 20:15:43.878810 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:43.878764 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="85caa676-1e13-4197-835f-eaaffb8d7d11" containerName="istio-proxy" Jul 06 20:15:43.882266 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:43.882247 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:43.884646 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:43.884627 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-6b6da4f1-openshift-default-dockercfg-msj4w\"" Jul 06 20:15:43.896170 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:43.896149 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr"] Jul 06 20:15:43.944163 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:43.944125 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-credential-socket\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:43.944357 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:43.944175 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/174cde53-45c1-45da-844d-f1e92938dc86-istio-podinfo\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:43.944357 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:43.944205 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/174cde53-45c1-45da-844d-f1e92938dc86-istio-token\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:43.944357 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:43.944273 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-workload-certs\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:43.944357 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:43.944353 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-istio-data\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:43.944531 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:43.944403 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-istio-envoy\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:43.944531 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:43.944485 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/174cde53-45c1-45da-844d-f1e92938dc86-istiod-ca-cert\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:43.944623 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:43.944534 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfhkg\" (UniqueName: \"kubernetes.io/projected/174cde53-45c1-45da-844d-f1e92938dc86-kube-api-access-cfhkg\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:43.944660 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:43.944620 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-workload-socket\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:44.045237 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.045189 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/174cde53-45c1-45da-844d-f1e92938dc86-istiod-ca-cert\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:44.045462 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.045249 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfhkg\" (UniqueName: \"kubernetes.io/projected/174cde53-45c1-45da-844d-f1e92938dc86-kube-api-access-cfhkg\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:44.045462 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.045308 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-workload-socket\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:44.045462 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.045368 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-credential-socket\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:44.045462 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.045392 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/174cde53-45c1-45da-844d-f1e92938dc86-istio-podinfo\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:44.045462 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.045420 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/174cde53-45c1-45da-844d-f1e92938dc86-istio-token\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:44.045462 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.045447 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-workload-certs\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:44.045793 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.045484 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-istio-data\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:44.045793 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.045528 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-istio-envoy\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:44.045904 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.045815 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-credential-socket\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:44.045904 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.045862 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-workload-socket\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:44.046072 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.046051 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/174cde53-45c1-45da-844d-f1e92938dc86-istiod-ca-cert\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:44.046141 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.046077 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-workload-certs\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:44.046141 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.046128 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-istio-data\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:44.048150 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.048129 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-istio-envoy\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:44.048235 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.048134 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/174cde53-45c1-45da-844d-f1e92938dc86-istio-podinfo\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:44.053085 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.053057 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfhkg\" (UniqueName: \"kubernetes.io/projected/174cde53-45c1-45da-844d-f1e92938dc86-kube-api-access-cfhkg\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:44.053544 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.053527 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/174cde53-45c1-45da-844d-f1e92938dc86-istio-token\") pod \"e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:44.193371 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.193273 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:44.334894 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.334869 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr"] Jul 06 20:15:44.336296 ip-10-0-130-172 kubenswrapper[2581]: W0706 20:15:44.336268 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod174cde53_45c1_45da_844d_f1e92938dc86.slice/crio-59ddee2b3f54f090f88e5aec0af5bf78b130bcf55365d0c815e42af803d0eb47 WatchSource:0}: Error finding container 59ddee2b3f54f090f88e5aec0af5bf78b130bcf55365d0c815e42af803d0eb47: Status 404 returned error can't find the container with id 59ddee2b3f54f090f88e5aec0af5bf78b130bcf55365d0c815e42af803d0eb47 Jul 06 20:15:44.338551 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.338516 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:15:44.338657 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.338598 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:15:44.338657 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:44.338645 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:15:45.211949 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:45.211917 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" event={"ID":"174cde53-45c1-45da-844d-f1e92938dc86","Type":"ContainerStarted","Data":"53f943ab5a7a67b977b70a29fc894ed00a1af01242991246bf7d0e42e42fdba3"} Jul 06 20:15:45.212407 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:45.211967 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" event={"ID":"174cde53-45c1-45da-844d-f1e92938dc86","Type":"ContainerStarted","Data":"59ddee2b3f54f090f88e5aec0af5bf78b130bcf55365d0c815e42af803d0eb47"} Jul 06 20:15:45.235564 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:45.235517 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" podStartSLOduration=2.235502203 podStartE2EDuration="2.235502203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:15:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:15:45.234287319 +0000 UTC m=+1644.681524038" watchObservedRunningTime="2026-07-06 20:15:45.235502203 +0000 UTC m=+1644.682738921" Jul 06 20:15:46.194366 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:46.194317 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:46.198978 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:46.198956 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:46.217314 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:46.217295 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:15:46.218013 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:15:46.217993 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:16:01.310951 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:01.309877 2581 patch_prober.go:28] interesting pod/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": read tcp 10.132.0.2:53474->10.132.0.49:15021: read: connection reset by peer" start-of-body= Jul 06 20:16:01.310951 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:01.309956 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" podUID="174cde53-45c1-45da-844d-f1e92938dc86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": read tcp 10.132.0.2:53474->10.132.0.49:15021: read: connection reset by peer" Jul 06 20:16:02.312012 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:02.311970 2581 generic.go:356] "Generic (PLEG): container finished" podID="174cde53-45c1-45da-844d-f1e92938dc86" containerID="53f943ab5a7a67b977b70a29fc894ed00a1af01242991246bf7d0e42e42fdba3" exitCode=0 Jul 06 20:16:02.312680 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:02.312652 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" event={"ID":"174cde53-45c1-45da-844d-f1e92938dc86","Type":"ContainerDied","Data":"53f943ab5a7a67b977b70a29fc894ed00a1af01242991246bf7d0e42e42fdba3"} Jul 06 20:16:02.313421 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:02.313399 2581 scope.go:117] "RemoveContainer" containerID="53f943ab5a7a67b977b70a29fc894ed00a1af01242991246bf7d0e42e42fdba3" Jul 06 20:16:02.314941 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:02.314912 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:16:02.315010 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:02.314975 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:16:02.315010 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:02.315003 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:16:03.319809 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:03.319772 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" event={"ID":"174cde53-45c1-45da-844d-f1e92938dc86","Type":"ContainerStarted","Data":"4370ced30a390180c460920c5b49ee0af6bbcb79982502f02d3658cb73b08338"} Jul 06 20:16:04.193649 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:04.193619 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:16:04.198224 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:04.198197 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:16:04.324573 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:04.324536 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:16:04.325537 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:04.325521 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:16:05.420357 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:05.420295 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr"] Jul 06 20:16:06.335303 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:06.335266 2581 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" secret="" err="secret \"e2e-mt-6b6da4f1-openshift-default-dockercfg-msj4w\" not found" Jul 06 20:16:07.340403 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:07.340368 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" podUID="174cde53-45c1-45da-844d-f1e92938dc86" containerName="istio-proxy" containerID="cri-o://4370ced30a390180c460920c5b49ee0af6bbcb79982502f02d3658cb73b08338" gracePeriod=30 Jul 06 20:16:12.600270 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.600246 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:16:12.621549 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.621432 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/174cde53-45c1-45da-844d-f1e92938dc86-istiod-ca-cert\") pod \"174cde53-45c1-45da-844d-f1e92938dc86\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " Jul 06 20:16:12.621549 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.621494 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-istio-envoy\") pod \"174cde53-45c1-45da-844d-f1e92938dc86\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " Jul 06 20:16:12.621549 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.621523 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/174cde53-45c1-45da-844d-f1e92938dc86-istio-token\") pod \"174cde53-45c1-45da-844d-f1e92938dc86\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " Jul 06 20:16:12.621549 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.621544 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-istio-data\") pod \"174cde53-45c1-45da-844d-f1e92938dc86\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " Jul 06 20:16:12.621867 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.621573 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfhkg\" (UniqueName: \"kubernetes.io/projected/174cde53-45c1-45da-844d-f1e92938dc86-kube-api-access-cfhkg\") pod \"174cde53-45c1-45da-844d-f1e92938dc86\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " Jul 06 20:16:12.621867 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.621637 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-workload-certs\") pod \"174cde53-45c1-45da-844d-f1e92938dc86\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " Jul 06 20:16:12.621867 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.621691 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/174cde53-45c1-45da-844d-f1e92938dc86-istio-podinfo\") pod \"174cde53-45c1-45da-844d-f1e92938dc86\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " Jul 06 20:16:12.621867 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.621732 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-credential-socket\") pod \"174cde53-45c1-45da-844d-f1e92938dc86\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " Jul 06 20:16:12.621867 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.621805 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-workload-socket\") pod \"174cde53-45c1-45da-844d-f1e92938dc86\" (UID: \"174cde53-45c1-45da-844d-f1e92938dc86\") " Jul 06 20:16:12.622760 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.622403 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "174cde53-45c1-45da-844d-f1e92938dc86" (UID: "174cde53-45c1-45da-844d-f1e92938dc86"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:16:12.622870 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.622837 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/174cde53-45c1-45da-844d-f1e92938dc86-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "174cde53-45c1-45da-844d-f1e92938dc86" (UID: "174cde53-45c1-45da-844d-f1e92938dc86"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:16:12.623097 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.623050 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "174cde53-45c1-45da-844d-f1e92938dc86" (UID: "174cde53-45c1-45da-844d-f1e92938dc86"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:16:12.623930 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.623901 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-istio-data" (OuterVolumeSpecName: "istio-data") pod "174cde53-45c1-45da-844d-f1e92938dc86" (UID: "174cde53-45c1-45da-844d-f1e92938dc86"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:16:12.624032 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.623973 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "174cde53-45c1-45da-844d-f1e92938dc86" (UID: "174cde53-45c1-45da-844d-f1e92938dc86"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:16:12.624820 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.624789 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "174cde53-45c1-45da-844d-f1e92938dc86" (UID: "174cde53-45c1-45da-844d-f1e92938dc86"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:16:12.624988 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.624959 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/174cde53-45c1-45da-844d-f1e92938dc86-kube-api-access-cfhkg" (OuterVolumeSpecName: "kube-api-access-cfhkg") pod "174cde53-45c1-45da-844d-f1e92938dc86" (UID: "174cde53-45c1-45da-844d-f1e92938dc86"). InnerVolumeSpecName "kube-api-access-cfhkg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:16:12.625075 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.625016 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/174cde53-45c1-45da-844d-f1e92938dc86-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "174cde53-45c1-45da-844d-f1e92938dc86" (UID: "174cde53-45c1-45da-844d-f1e92938dc86"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:16:12.625462 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.625444 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/174cde53-45c1-45da-844d-f1e92938dc86-istio-token" (OuterVolumeSpecName: "istio-token") pod "174cde53-45c1-45da-844d-f1e92938dc86" (UID: "174cde53-45c1-45da-844d-f1e92938dc86"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:16:12.722827 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.722797 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/174cde53-45c1-45da-844d-f1e92938dc86-istiod-ca-cert\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:16:12.722827 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.722829 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-istio-envoy\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:16:12.723031 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.722842 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/174cde53-45c1-45da-844d-f1e92938dc86-istio-token\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:16:12.723031 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.722854 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-istio-data\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:16:12.723031 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.722868 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfhkg\" (UniqueName: \"kubernetes.io/projected/174cde53-45c1-45da-844d-f1e92938dc86-kube-api-access-cfhkg\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:16:12.723031 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.722883 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-workload-certs\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:16:12.723031 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.722897 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/174cde53-45c1-45da-844d-f1e92938dc86-istio-podinfo\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:16:12.723031 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.722909 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-credential-socket\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:16:12.723031 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:12.722921 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/174cde53-45c1-45da-844d-f1e92938dc86-workload-socket\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:16:13.373412 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:13.373302 2581 generic.go:356] "Generic (PLEG): container finished" podID="174cde53-45c1-45da-844d-f1e92938dc86" containerID="4370ced30a390180c460920c5b49ee0af6bbcb79982502f02d3658cb73b08338" exitCode=0 Jul 06 20:16:13.373412 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:13.373379 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" Jul 06 20:16:13.373412 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:13.373391 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" event={"ID":"174cde53-45c1-45da-844d-f1e92938dc86","Type":"ContainerDied","Data":"4370ced30a390180c460920c5b49ee0af6bbcb79982502f02d3658cb73b08338"} Jul 06 20:16:13.373636 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:13.373452 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr" event={"ID":"174cde53-45c1-45da-844d-f1e92938dc86","Type":"ContainerDied","Data":"59ddee2b3f54f090f88e5aec0af5bf78b130bcf55365d0c815e42af803d0eb47"} Jul 06 20:16:13.373636 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:13.373474 2581 scope.go:117] "RemoveContainer" containerID="4370ced30a390180c460920c5b49ee0af6bbcb79982502f02d3658cb73b08338" Jul 06 20:16:13.382261 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:13.382247 2581 scope.go:117] "RemoveContainer" containerID="53f943ab5a7a67b977b70a29fc894ed00a1af01242991246bf7d0e42e42fdba3" Jul 06 20:16:13.390788 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:13.390759 2581 scope.go:117] "RemoveContainer" containerID="4370ced30a390180c460920c5b49ee0af6bbcb79982502f02d3658cb73b08338" Jul 06 20:16:13.391154 ip-10-0-130-172 kubenswrapper[2581]: E0706 20:16:13.391117 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4370ced30a390180c460920c5b49ee0af6bbcb79982502f02d3658cb73b08338\": container with ID starting with 4370ced30a390180c460920c5b49ee0af6bbcb79982502f02d3658cb73b08338 not found: ID does not exist" containerID="4370ced30a390180c460920c5b49ee0af6bbcb79982502f02d3658cb73b08338" Jul 06 20:16:13.391251 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:13.391166 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4370ced30a390180c460920c5b49ee0af6bbcb79982502f02d3658cb73b08338"} err="failed to get container status \"4370ced30a390180c460920c5b49ee0af6bbcb79982502f02d3658cb73b08338\": rpc error: code = NotFound desc = could not find container \"4370ced30a390180c460920c5b49ee0af6bbcb79982502f02d3658cb73b08338\": container with ID starting with 4370ced30a390180c460920c5b49ee0af6bbcb79982502f02d3658cb73b08338 not found: ID does not exist" Jul 06 20:16:13.391251 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:13.391192 2581 scope.go:117] "RemoveContainer" containerID="53f943ab5a7a67b977b70a29fc894ed00a1af01242991246bf7d0e42e42fdba3" Jul 06 20:16:13.391613 ip-10-0-130-172 kubenswrapper[2581]: E0706 20:16:13.391591 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53f943ab5a7a67b977b70a29fc894ed00a1af01242991246bf7d0e42e42fdba3\": container with ID starting with 53f943ab5a7a67b977b70a29fc894ed00a1af01242991246bf7d0e42e42fdba3 not found: ID does not exist" containerID="53f943ab5a7a67b977b70a29fc894ed00a1af01242991246bf7d0e42e42fdba3" Jul 06 20:16:13.391696 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:13.391621 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53f943ab5a7a67b977b70a29fc894ed00a1af01242991246bf7d0e42e42fdba3"} err="failed to get container status \"53f943ab5a7a67b977b70a29fc894ed00a1af01242991246bf7d0e42e42fdba3\": rpc error: code = NotFound desc = could not find container \"53f943ab5a7a67b977b70a29fc894ed00a1af01242991246bf7d0e42e42fdba3\": container with ID starting with 53f943ab5a7a67b977b70a29fc894ed00a1af01242991246bf7d0e42e42fdba3 not found: ID does not exist" Jul 06 20:16:13.398070 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:13.398044 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr"] Jul 06 20:16:13.400627 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:13.400605 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6b6da4f1-openshift-default-685b7ccfc9-mz6dr"] Jul 06 20:16:15.094602 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:15.094568 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="174cde53-45c1-45da-844d-f1e92938dc86" path="/var/lib/kubelet/pods/174cde53-45c1-45da-844d-f1e92938dc86/volumes" Jul 06 20:16:25.223015 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.222982 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5"] Jul 06 20:16:25.223659 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.223639 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="174cde53-45c1-45da-844d-f1e92938dc86" containerName="istio-proxy" Jul 06 20:16:25.223712 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.223663 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="174cde53-45c1-45da-844d-f1e92938dc86" containerName="istio-proxy" Jul 06 20:16:25.223785 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.223774 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="174cde53-45c1-45da-844d-f1e92938dc86" containerName="istio-proxy" Jul 06 20:16:25.227116 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.227099 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.247041 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.247003 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2bec5f66-openshift-default-dockercfg-wlwkh\"" Jul 06 20:16:25.258553 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.258523 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5"] Jul 06 20:16:25.344417 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.344378 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83b865ba-9449-47f1-af75-b8a67a39351f-istio-token\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.344595 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.344441 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83b865ba-9449-47f1-af75-b8a67a39351f-istio-podinfo\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.344595 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.344509 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83b865ba-9449-47f1-af75-b8a67a39351f-istiod-ca-cert\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.344595 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.344544 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-istio-data\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.344595 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.344567 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwg2d\" (UniqueName: \"kubernetes.io/projected/83b865ba-9449-47f1-af75-b8a67a39351f-kube-api-access-lwg2d\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.344595 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.344591 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-credential-socket\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.344860 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.344698 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-istio-envoy\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.344860 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.344741 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-workload-socket\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.344860 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.344844 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-workload-certs\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.445879 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.445844 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83b865ba-9449-47f1-af75-b8a67a39351f-istio-token\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.445879 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.445885 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83b865ba-9449-47f1-af75-b8a67a39351f-istio-podinfo\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.446133 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.445996 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83b865ba-9449-47f1-af75-b8a67a39351f-istiod-ca-cert\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.446133 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.446034 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-istio-data\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.446133 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.446059 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwg2d\" (UniqueName: \"kubernetes.io/projected/83b865ba-9449-47f1-af75-b8a67a39351f-kube-api-access-lwg2d\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.446306 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.446165 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-credential-socket\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.446306 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.446224 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-istio-envoy\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.446453 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.446361 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-workload-socket\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.446453 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.446433 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-workload-certs\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.446561 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.446461 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-istio-data\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.446616 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.446597 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-credential-socket\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.446782 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.446718 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-workload-socket\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.446782 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.446763 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-workload-certs\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.446992 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.446872 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83b865ba-9449-47f1-af75-b8a67a39351f-istiod-ca-cert\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.448422 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.448399 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-istio-envoy\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.448660 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.448641 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83b865ba-9449-47f1-af75-b8a67a39351f-istio-podinfo\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.453934 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.453915 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83b865ba-9449-47f1-af75-b8a67a39351f-istio-token\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.454333 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.454300 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwg2d\" (UniqueName: \"kubernetes.io/projected/83b865ba-9449-47f1-af75-b8a67a39351f-kube-api-access-lwg2d\") pod \"e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.536860 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.536829 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:25.683864 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.683838 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5"] Jul 06 20:16:25.685458 ip-10-0-130-172 kubenswrapper[2581]: W0706 20:16:25.685429 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83b865ba_9449_47f1_af75_b8a67a39351f.slice/crio-58fdc4a17ed58fba3054a5c218adce1b0b44646d0bb4af6156f753fb42f63971 WatchSource:0}: Error finding container 58fdc4a17ed58fba3054a5c218adce1b0b44646d0bb4af6156f753fb42f63971: Status 404 returned error can't find the container with id 58fdc4a17ed58fba3054a5c218adce1b0b44646d0bb4af6156f753fb42f63971 Jul 06 20:16:25.687559 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.687528 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:16:25.687651 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.687599 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:16:25.687651 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:25.687630 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:16:26.441513 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:26.441469 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" event={"ID":"83b865ba-9449-47f1-af75-b8a67a39351f","Type":"ContainerStarted","Data":"8cafe179e372941f155671374018a14563bf0e0833651e5a3fa5d6235e5b5c0b"} Jul 06 20:16:26.441927 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:26.441542 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" event={"ID":"83b865ba-9449-47f1-af75-b8a67a39351f","Type":"ContainerStarted","Data":"58fdc4a17ed58fba3054a5c218adce1b0b44646d0bb4af6156f753fb42f63971"} Jul 06 20:16:26.465076 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:26.465028 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" podStartSLOduration=1.465013157 podStartE2EDuration="1.465013157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:16:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:16:26.463903859 +0000 UTC m=+1685.911140654" watchObservedRunningTime="2026-07-06 20:16:26.465013157 +0000 UTC m=+1685.912249935" Jul 06 20:16:26.537172 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:26.537135 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:26.541978 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:26.541954 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:27.446755 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:27.446717 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:27.447946 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:27.447922 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:41.520200 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:41.520168 2581 generic.go:356] "Generic (PLEG): container finished" podID="83b865ba-9449-47f1-af75-b8a67a39351f" containerID="8cafe179e372941f155671374018a14563bf0e0833651e5a3fa5d6235e5b5c0b" exitCode=0 Jul 06 20:16:41.520718 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:41.520220 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" event={"ID":"83b865ba-9449-47f1-af75-b8a67a39351f","Type":"ContainerDied","Data":"8cafe179e372941f155671374018a14563bf0e0833651e5a3fa5d6235e5b5c0b"} Jul 06 20:16:41.520718 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:41.520630 2581 scope.go:117] "RemoveContainer" containerID="8cafe179e372941f155671374018a14563bf0e0833651e5a3fa5d6235e5b5c0b" Jul 06 20:16:41.521711 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:41.521675 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:16:41.521805 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:41.521753 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:16:41.521805 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:41.521787 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:16:42.447675 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:42.447643 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:42.526642 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:42.526611 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" event={"ID":"83b865ba-9449-47f1-af75-b8a67a39351f","Type":"ContainerStarted","Data":"6b823ebfcb4b536bc01556bb8fcbc0261549f74b1c5a21690f57d31d4f08082a"} Jul 06 20:16:42.537306 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:42.537286 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:42.538354 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:42.538331 2581 patch_prober.go:28] interesting pod/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 06 20:16:42.538464 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:42.538373 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" podUID="83b865ba-9449-47f1-af75-b8a67a39351f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 06 20:16:43.537241 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:43.537167 2581 patch_prober.go:28] interesting pod/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 06 20:16:43.537241 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:43.537216 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" podUID="83b865ba-9449-47f1-af75-b8a67a39351f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 06 20:16:44.537261 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:44.537227 2581 patch_prober.go:28] interesting pod/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 06 20:16:44.537727 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:44.537285 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" podUID="83b865ba-9449-47f1-af75-b8a67a39351f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 06 20:16:45.537492 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:45.537447 2581 patch_prober.go:28] interesting pod/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 06 20:16:45.537938 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:45.537530 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" podUID="83b865ba-9449-47f1-af75-b8a67a39351f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 06 20:16:46.568805 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:46.568774 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:46.569244 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:46.569164 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:16:46.571542 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:16:46.571518 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:17:54.692146 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:17:54.692112 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5"] Jul 06 20:17:54.692672 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:17:54.692387 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" podUID="83b865ba-9449-47f1-af75-b8a67a39351f" containerName="istio-proxy" containerID="cri-o://6b823ebfcb4b536bc01556bb8fcbc0261549f74b1c5a21690f57d31d4f08082a" gracePeriod=30 Jul 06 20:17:59.936086 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:17:59.936055 2581 generic.go:356] "Generic (PLEG): container finished" podID="83b865ba-9449-47f1-af75-b8a67a39351f" containerID="6b823ebfcb4b536bc01556bb8fcbc0261549f74b1c5a21690f57d31d4f08082a" exitCode=0 Jul 06 20:17:59.936472 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:17:59.936108 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" event={"ID":"83b865ba-9449-47f1-af75-b8a67a39351f","Type":"ContainerDied","Data":"6b823ebfcb4b536bc01556bb8fcbc0261549f74b1c5a21690f57d31d4f08082a"} Jul 06 20:17:59.936472 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:17:59.936179 2581 scope.go:117] "RemoveContainer" containerID="8cafe179e372941f155671374018a14563bf0e0833651e5a3fa5d6235e5b5c0b" Jul 06 20:17:59.973504 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:17:59.973483 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:18:00.012580 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.012539 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83b865ba-9449-47f1-af75-b8a67a39351f-istio-podinfo\") pod \"83b865ba-9449-47f1-af75-b8a67a39351f\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " Jul 06 20:18:00.012736 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.012610 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83b865ba-9449-47f1-af75-b8a67a39351f-istio-token\") pod \"83b865ba-9449-47f1-af75-b8a67a39351f\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " Jul 06 20:18:00.012736 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.012642 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83b865ba-9449-47f1-af75-b8a67a39351f-istiod-ca-cert\") pod \"83b865ba-9449-47f1-af75-b8a67a39351f\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " Jul 06 20:18:00.012736 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.012691 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-credential-socket\") pod \"83b865ba-9449-47f1-af75-b8a67a39351f\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " Jul 06 20:18:00.012736 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.012728 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-istio-data\") pod \"83b865ba-9449-47f1-af75-b8a67a39351f\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " Jul 06 20:18:00.013045 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.013013 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "83b865ba-9449-47f1-af75-b8a67a39351f" (UID: "83b865ba-9449-47f1-af75-b8a67a39351f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:18:00.013162 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.013056 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83b865ba-9449-47f1-af75-b8a67a39351f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "83b865ba-9449-47f1-af75-b8a67a39351f" (UID: "83b865ba-9449-47f1-af75-b8a67a39351f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:18:00.013386 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.012780 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwg2d\" (UniqueName: \"kubernetes.io/projected/83b865ba-9449-47f1-af75-b8a67a39351f-kube-api-access-lwg2d\") pod \"83b865ba-9449-47f1-af75-b8a67a39351f\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " Jul 06 20:18:00.013591 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.013571 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-istio-envoy\") pod \"83b865ba-9449-47f1-af75-b8a67a39351f\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " Jul 06 20:18:00.013725 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.013699 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-istio-data" (OuterVolumeSpecName: "istio-data") pod "83b865ba-9449-47f1-af75-b8a67a39351f" (UID: "83b865ba-9449-47f1-af75-b8a67a39351f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:18:00.013957 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.013718 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-workload-socket\") pod \"83b865ba-9449-47f1-af75-b8a67a39351f\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " Jul 06 20:18:00.014148 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.013954 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "83b865ba-9449-47f1-af75-b8a67a39351f" (UID: "83b865ba-9449-47f1-af75-b8a67a39351f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:18:00.014257 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.014129 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-workload-certs\") pod \"83b865ba-9449-47f1-af75-b8a67a39351f\" (UID: \"83b865ba-9449-47f1-af75-b8a67a39351f\") " Jul 06 20:18:00.014366 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.014315 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "83b865ba-9449-47f1-af75-b8a67a39351f" (UID: "83b865ba-9449-47f1-af75-b8a67a39351f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:18:00.014600 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.014574 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-workload-socket\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:18:00.014680 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.014603 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-workload-certs\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:18:00.014680 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.014619 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83b865ba-9449-47f1-af75-b8a67a39351f-istiod-ca-cert\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:18:00.014680 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.014633 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-credential-socket\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:18:00.014680 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.014647 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-istio-data\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:18:00.015063 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.015045 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83b865ba-9449-47f1-af75-b8a67a39351f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "83b865ba-9449-47f1-af75-b8a67a39351f" (UID: "83b865ba-9449-47f1-af75-b8a67a39351f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:18:00.015130 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.015064 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83b865ba-9449-47f1-af75-b8a67a39351f-istio-token" (OuterVolumeSpecName: "istio-token") pod "83b865ba-9449-47f1-af75-b8a67a39351f" (UID: "83b865ba-9449-47f1-af75-b8a67a39351f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:18:00.015920 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.015892 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83b865ba-9449-47f1-af75-b8a67a39351f-kube-api-access-lwg2d" (OuterVolumeSpecName: "kube-api-access-lwg2d") pod "83b865ba-9449-47f1-af75-b8a67a39351f" (UID: "83b865ba-9449-47f1-af75-b8a67a39351f"). InnerVolumeSpecName "kube-api-access-lwg2d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:18:00.015989 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.015936 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "83b865ba-9449-47f1-af75-b8a67a39351f" (UID: "83b865ba-9449-47f1-af75-b8a67a39351f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:18:00.116013 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.115978 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83b865ba-9449-47f1-af75-b8a67a39351f-istio-token\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:18:00.116013 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.116007 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwg2d\" (UniqueName: \"kubernetes.io/projected/83b865ba-9449-47f1-af75-b8a67a39351f-kube-api-access-lwg2d\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:18:00.116013 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.116020 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83b865ba-9449-47f1-af75-b8a67a39351f-istio-envoy\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:18:00.116245 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.116027 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83b865ba-9449-47f1-af75-b8a67a39351f-istio-podinfo\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:18:00.941916 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.941887 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" Jul 06 20:18:00.941916 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.941910 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5" event={"ID":"83b865ba-9449-47f1-af75-b8a67a39351f","Type":"ContainerDied","Data":"58fdc4a17ed58fba3054a5c218adce1b0b44646d0bb4af6156f753fb42f63971"} Jul 06 20:18:00.942414 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.941968 2581 scope.go:117] "RemoveContainer" containerID="6b823ebfcb4b536bc01556bb8fcbc0261549f74b1c5a21690f57d31d4f08082a" Jul 06 20:18:00.966557 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.966531 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5"] Jul 06 20:18:00.970183 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:00.970161 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2bec5f66-openshift-default-6b58b699cf-5xxr5"] Jul 06 20:18:01.095850 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:01.095819 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83b865ba-9449-47f1-af75-b8a67a39351f" path="/var/lib/kubelet/pods/83b865ba-9449-47f1-af75-b8a67a39351f/volumes" Jul 06 20:18:21.150150 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:21.150123 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/2.log" Jul 06 20:18:21.156426 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:21.156405 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/2.log" Jul 06 20:18:49.791226 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:49.791194 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649"] Jul 06 20:18:49.791706 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:49.791690 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83b865ba-9449-47f1-af75-b8a67a39351f" containerName="istio-proxy" Jul 06 20:18:49.791751 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:49.791709 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="83b865ba-9449-47f1-af75-b8a67a39351f" containerName="istio-proxy" Jul 06 20:18:49.791751 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:49.791739 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="174cde53-45c1-45da-844d-f1e92938dc86" containerName="istio-proxy" Jul 06 20:18:49.791751 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:49.791749 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="174cde53-45c1-45da-844d-f1e92938dc86" containerName="istio-proxy" Jul 06 20:18:49.791887 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:49.791876 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="83b865ba-9449-47f1-af75-b8a67a39351f" containerName="istio-proxy" Jul 06 20:18:49.791924 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:49.791894 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="83b865ba-9449-47f1-af75-b8a67a39351f" containerName="istio-proxy" Jul 06 20:18:49.791924 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:49.791906 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="174cde53-45c1-45da-844d-f1e92938dc86" containerName="istio-proxy" Jul 06 20:18:49.794672 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:49.794657 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:49.797931 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:49.797915 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-e7dc01-openshift-default-dockercfg-pcbh7\"" Jul 06 20:18:49.812223 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:49.812199 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649"] Jul 06 20:18:49.904705 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:49.904667 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-workload-certs\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:49.904705 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:49.904707 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db854180-8b62-48e3-ad7d-396518053ad7-istio-token\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:49.904923 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:49.904736 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-credential-socket\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:49.904923 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:49.904758 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db854180-8b62-48e3-ad7d-396518053ad7-istiod-ca-cert\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:49.904923 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:49.904787 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-istio-data\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:49.904923 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:49.904851 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-workload-socket\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:49.904923 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:49.904896 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db854180-8b62-48e3-ad7d-396518053ad7-istio-podinfo\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:49.904923 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:49.904920 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-istio-envoy\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:49.905178 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:49.904940 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxmp9\" (UniqueName: \"kubernetes.io/projected/db854180-8b62-48e3-ad7d-396518053ad7-kube-api-access-nxmp9\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:50.005920 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.005882 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db854180-8b62-48e3-ad7d-396518053ad7-istio-podinfo\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:50.005920 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.005928 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-istio-envoy\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:50.006190 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.005946 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxmp9\" (UniqueName: \"kubernetes.io/projected/db854180-8b62-48e3-ad7d-396518053ad7-kube-api-access-nxmp9\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:50.006190 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.005991 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-workload-certs\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:50.006190 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.006006 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db854180-8b62-48e3-ad7d-396518053ad7-istio-token\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:50.006190 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.006023 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-credential-socket\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:50.006190 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.006037 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db854180-8b62-48e3-ad7d-396518053ad7-istiod-ca-cert\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:50.006190 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.006060 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-istio-data\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:50.006190 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.006112 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-workload-socket\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:50.006600 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.006512 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-credential-socket\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:50.006600 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.006546 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-workload-certs\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:50.006708 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.006595 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-workload-socket\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:50.006708 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.006670 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-istio-data\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:50.006912 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.006889 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db854180-8b62-48e3-ad7d-396518053ad7-istiod-ca-cert\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:50.008198 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.008176 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-istio-envoy\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:50.008535 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.008517 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db854180-8b62-48e3-ad7d-396518053ad7-istio-podinfo\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:50.013600 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.013573 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db854180-8b62-48e3-ad7d-396518053ad7-istio-token\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:50.013714 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.013699 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxmp9\" (UniqueName: \"kubernetes.io/projected/db854180-8b62-48e3-ad7d-396518053ad7-kube-api-access-nxmp9\") pod \"e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:50.106182 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.106093 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:50.252437 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.252408 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649"] Jul 06 20:18:50.254181 ip-10-0-130-172 kubenswrapper[2581]: W0706 20:18:50.254147 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb854180_8b62_48e3_ad7d_396518053ad7.slice/crio-3c1c2ba08d9d131f8620ca4cde5ce9ca0c816afc298a016a52c9066c05af18b3 WatchSource:0}: Error finding container 3c1c2ba08d9d131f8620ca4cde5ce9ca0c816afc298a016a52c9066c05af18b3: Status 404 returned error can't find the container with id 3c1c2ba08d9d131f8620ca4cde5ce9ca0c816afc298a016a52c9066c05af18b3 Jul 06 20:18:50.256845 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.256813 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:18:50.256954 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.256881 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:18:50.256954 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:50.256914 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:18:51.211238 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:51.211193 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" event={"ID":"db854180-8b62-48e3-ad7d-396518053ad7","Type":"ContainerStarted","Data":"453b04fcd47b47acb386089c597246a6ef1c466c984592b7c81ee3384b219f40"} Jul 06 20:18:51.211638 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:51.211263 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" event={"ID":"db854180-8b62-48e3-ad7d-396518053ad7","Type":"ContainerStarted","Data":"3c1c2ba08d9d131f8620ca4cde5ce9ca0c816afc298a016a52c9066c05af18b3"} Jul 06 20:18:51.233600 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:51.233555 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" podStartSLOduration=2.233539347 podStartE2EDuration="2.233539347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:18:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:18:51.231493748 +0000 UTC m=+1830.678730468" watchObservedRunningTime="2026-07-06 20:18:51.233539347 +0000 UTC m=+1830.680776126" Jul 06 20:18:52.106403 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:52.106368 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:52.111289 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:52.111267 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:52.216878 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:52.216844 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:52.217691 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:52.217674 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:18:58.372552 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.372519 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4"] Jul 06 20:18:58.373158 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.373138 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83b865ba-9449-47f1-af75-b8a67a39351f" containerName="istio-proxy" Jul 06 20:18:58.373158 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.373159 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="83b865ba-9449-47f1-af75-b8a67a39351f" containerName="istio-proxy" Jul 06 20:18:58.376347 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.376286 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.378982 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.378959 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ec5878-openshift-default-dockercfg-jmzqs\"" Jul 06 20:18:58.392000 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.391974 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4"] Jul 06 20:18:58.491276 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.491242 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istiod-ca-cert\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.491484 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.491377 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-podinfo\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.491484 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.491440 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-workload-certs\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.491484 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.491463 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-token\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.491644 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.491509 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-credential-socket\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.491644 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.491559 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-data\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.491736 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.491633 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzdl2\" (UniqueName: \"kubernetes.io/projected/caaa1ed6-661e-4dd7-9e9b-151af3aba580-kube-api-access-qzdl2\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.491736 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.491676 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-envoy\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.491736 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.491722 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-workload-socket\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.592912 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.592876 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istiod-ca-cert\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.593095 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.592935 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-podinfo\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.593095 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.592973 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-workload-certs\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.593095 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.593000 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-token\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.593095 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.593048 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-credential-socket\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.593095 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.593090 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-data\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.593423 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.593119 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzdl2\" (UniqueName: \"kubernetes.io/projected/caaa1ed6-661e-4dd7-9e9b-151af3aba580-kube-api-access-qzdl2\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.593423 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.593144 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-envoy\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.593423 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.593173 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-workload-socket\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.593423 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.593432 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-workload-certs\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.593641 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.593539 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-data\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.593742 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.593671 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istiod-ca-cert\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.593742 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.593732 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-credential-socket\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.594019 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.593813 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-workload-socket\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.595478 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.595460 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-envoy\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.595865 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.595843 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-podinfo\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.603894 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.603867 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-token\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.604005 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.603987 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzdl2\" (UniqueName: \"kubernetes.io/projected/caaa1ed6-661e-4dd7-9e9b-151af3aba580-kube-api-access-qzdl2\") pod \"e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.688632 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.688538 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:58.830944 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.830921 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4"] Jul 06 20:18:58.833068 ip-10-0-130-172 kubenswrapper[2581]: W0706 20:18:58.833038 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaaa1ed6_661e_4dd7_9e9b_151af3aba580.slice/crio-0a70bbd6ec6c6bdd125efbcc22e29601f34043510ad37b9431496d9df19862f6 WatchSource:0}: Error finding container 0a70bbd6ec6c6bdd125efbcc22e29601f34043510ad37b9431496d9df19862f6: Status 404 returned error can't find the container with id 0a70bbd6ec6c6bdd125efbcc22e29601f34043510ad37b9431496d9df19862f6 Jul 06 20:18:58.835631 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.835601 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:18:58.835697 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.835673 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:18:58.835735 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:58.835704 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:18:59.258498 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:59.258464 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" event={"ID":"caaa1ed6-661e-4dd7-9e9b-151af3aba580","Type":"ContainerStarted","Data":"d7b4f90b42b73637f5698d5d175427309adcda4643e0a8526d559bb2e767c2c5"} Jul 06 20:18:59.258683 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:59.258515 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" event={"ID":"caaa1ed6-661e-4dd7-9e9b-151af3aba580","Type":"ContainerStarted","Data":"0a70bbd6ec6c6bdd125efbcc22e29601f34043510ad37b9431496d9df19862f6"} Jul 06 20:18:59.284129 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:59.284084 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" podStartSLOduration=1.284069451 podStartE2EDuration="1.284069451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:18:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:18:59.282076666 +0000 UTC m=+1838.729313387" watchObservedRunningTime="2026-07-06 20:18:59.284069451 +0000 UTC m=+1838.731306169" Jul 06 20:18:59.689153 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:59.689061 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:18:59.694146 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:18:59.694120 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:19:00.263837 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:00.263809 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:19:00.264802 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:00.264784 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:19:08.258515 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:08.258466 2581 patch_prober.go:28] interesting pod/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 20:19:08.258978 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:08.258540 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" podUID="db854180-8b62-48e3-ad7d-396518053ad7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 20:19:09.316033 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:09.316001 2581 generic.go:356] "Generic (PLEG): container finished" podID="db854180-8b62-48e3-ad7d-396518053ad7" containerID="453b04fcd47b47acb386089c597246a6ef1c466c984592b7c81ee3384b219f40" exitCode=0 Jul 06 20:19:09.316501 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:09.316076 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" event={"ID":"db854180-8b62-48e3-ad7d-396518053ad7","Type":"ContainerDied","Data":"453b04fcd47b47acb386089c597246a6ef1c466c984592b7c81ee3384b219f40"} Jul 06 20:19:09.316587 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:09.316563 2581 scope.go:117] "RemoveContainer" containerID="453b04fcd47b47acb386089c597246a6ef1c466c984592b7c81ee3384b219f40" Jul 06 20:19:09.317586 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:09.317563 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:19:09.317657 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:09.317623 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:19:09.317694 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:09.317662 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:19:10.322768 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:10.322734 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" event={"ID":"db854180-8b62-48e3-ad7d-396518053ad7","Type":"ContainerStarted","Data":"03962e01b2b8f92de920ffbd8cd21ac073dc0223bab00209c59d8eebfabbee7d"} Jul 06 20:19:11.106705 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:11.106673 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:19:11.106908 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:11.106887 2581 patch_prober.go:28] interesting pod/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 06 20:19:11.106971 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:11.106946 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" podUID="db854180-8b62-48e3-ad7d-396518053ad7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 06 20:19:12.107443 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:12.107409 2581 patch_prober.go:28] interesting pod/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 06 20:19:12.107837 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:12.107487 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" podUID="db854180-8b62-48e3-ad7d-396518053ad7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 06 20:19:13.107195 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:13.107163 2581 patch_prober.go:28] interesting pod/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 06 20:19:13.107383 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:13.107220 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" podUID="db854180-8b62-48e3-ad7d-396518053ad7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 06 20:19:14.107390 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:14.107348 2581 patch_prober.go:28] interesting pod/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 06 20:19:14.107776 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:14.107432 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" podUID="db854180-8b62-48e3-ad7d-396518053ad7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 06 20:19:15.109997 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:15.109962 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:19:15.110499 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:15.110348 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:19:15.111260 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:15.111241 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:19:16.306518 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:16.306471 2581 patch_prober.go:28] interesting pod/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 20:19:16.307057 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:16.306559 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" podUID="caaa1ed6-661e-4dd7-9e9b-151af3aba580" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 20:19:17.367526 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:17.367491 2581 generic.go:356] "Generic (PLEG): container finished" podID="caaa1ed6-661e-4dd7-9e9b-151af3aba580" containerID="d7b4f90b42b73637f5698d5d175427309adcda4643e0a8526d559bb2e767c2c5" exitCode=0 Jul 06 20:19:17.367941 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:17.367572 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" event={"ID":"caaa1ed6-661e-4dd7-9e9b-151af3aba580","Type":"ContainerDied","Data":"d7b4f90b42b73637f5698d5d175427309adcda4643e0a8526d559bb2e767c2c5"} Jul 06 20:19:17.368159 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:17.368143 2581 scope.go:117] "RemoveContainer" containerID="d7b4f90b42b73637f5698d5d175427309adcda4643e0a8526d559bb2e767c2c5" Jul 06 20:19:17.369429 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:17.369399 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:19:17.369550 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:17.369485 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:19:17.369550 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:17.369528 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:19:18.379397 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:18.378432 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" event={"ID":"caaa1ed6-661e-4dd7-9e9b-151af3aba580","Type":"ContainerStarted","Data":"d4a03277db70e6707cdb1ae41a33985d0bd0f08671a7feaf7142053a50036241"} Jul 06 20:19:18.689659 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:18.689563 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:19:18.689831 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:18.689809 2581 patch_prober.go:28] interesting pod/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 06 20:19:18.689875 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:18.689858 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" podUID="caaa1ed6-661e-4dd7-9e9b-151af3aba580" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 06 20:19:19.689737 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:19.689698 2581 patch_prober.go:28] interesting pod/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 06 20:19:19.690159 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:19.689753 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" podUID="caaa1ed6-661e-4dd7-9e9b-151af3aba580" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 06 20:19:20.689402 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:20.689365 2581 patch_prober.go:28] interesting pod/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 06 20:19:20.689608 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:20.689424 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" podUID="caaa1ed6-661e-4dd7-9e9b-151af3aba580" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 06 20:19:21.689850 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:21.689811 2581 patch_prober.go:28] interesting pod/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 06 20:19:21.690243 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:21.689883 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" podUID="caaa1ed6-661e-4dd7-9e9b-151af3aba580" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 06 20:19:22.693228 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:22.693198 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:19:22.693644 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:22.693552 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:19:22.694186 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:19:22.694170 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:21:08.390714 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:08.390672 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649"] Jul 06 20:21:08.391304 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:08.390944 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" podUID="db854180-8b62-48e3-ad7d-396518053ad7" containerName="istio-proxy" containerID="cri-o://03962e01b2b8f92de920ffbd8cd21ac073dc0223bab00209c59d8eebfabbee7d" gracePeriod=30 Jul 06 20:21:13.649208 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.649184 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:21:13.746425 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.746310 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db854180-8b62-48e3-ad7d-396518053ad7-istio-token\") pod \"db854180-8b62-48e3-ad7d-396518053ad7\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " Jul 06 20:21:13.746425 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.746389 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxmp9\" (UniqueName: \"kubernetes.io/projected/db854180-8b62-48e3-ad7d-396518053ad7-kube-api-access-nxmp9\") pod \"db854180-8b62-48e3-ad7d-396518053ad7\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " Jul 06 20:21:13.746670 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.746430 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-workload-certs\") pod \"db854180-8b62-48e3-ad7d-396518053ad7\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " Jul 06 20:21:13.746670 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.746464 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-credential-socket\") pod \"db854180-8b62-48e3-ad7d-396518053ad7\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " Jul 06 20:21:13.746670 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.746498 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-istio-data\") pod \"db854180-8b62-48e3-ad7d-396518053ad7\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " Jul 06 20:21:13.746670 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.746558 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-workload-socket\") pod \"db854180-8b62-48e3-ad7d-396518053ad7\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " Jul 06 20:21:13.746670 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.746621 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-istio-envoy\") pod \"db854180-8b62-48e3-ad7d-396518053ad7\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " Jul 06 20:21:13.746924 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.746848 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db854180-8b62-48e3-ad7d-396518053ad7-istiod-ca-cert\") pod \"db854180-8b62-48e3-ad7d-396518053ad7\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " Jul 06 20:21:13.746924 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.746889 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db854180-8b62-48e3-ad7d-396518053ad7-istio-podinfo\") pod \"db854180-8b62-48e3-ad7d-396518053ad7\" (UID: \"db854180-8b62-48e3-ad7d-396518053ad7\") " Jul 06 20:21:13.746924 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.746885 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "db854180-8b62-48e3-ad7d-396518053ad7" (UID: "db854180-8b62-48e3-ad7d-396518053ad7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:21:13.747085 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.746920 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "db854180-8b62-48e3-ad7d-396518053ad7" (UID: "db854180-8b62-48e3-ad7d-396518053ad7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:21:13.747251 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.747202 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db854180-8b62-48e3-ad7d-396518053ad7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "db854180-8b62-48e3-ad7d-396518053ad7" (UID: "db854180-8b62-48e3-ad7d-396518053ad7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:21:13.747363 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.747311 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-workload-socket\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:13.747431 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.747370 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db854180-8b62-48e3-ad7d-396518053ad7-istiod-ca-cert\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:13.747431 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.747385 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-credential-socket\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:13.748046 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.748024 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-istio-data" (OuterVolumeSpecName: "istio-data") pod "db854180-8b62-48e3-ad7d-396518053ad7" (UID: "db854180-8b62-48e3-ad7d-396518053ad7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:21:13.748512 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.748489 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "db854180-8b62-48e3-ad7d-396518053ad7" (UID: "db854180-8b62-48e3-ad7d-396518053ad7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:21:13.749192 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.749159 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "db854180-8b62-48e3-ad7d-396518053ad7" (UID: "db854180-8b62-48e3-ad7d-396518053ad7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:21:13.749192 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.749172 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db854180-8b62-48e3-ad7d-396518053ad7-kube-api-access-nxmp9" (OuterVolumeSpecName: "kube-api-access-nxmp9") pod "db854180-8b62-48e3-ad7d-396518053ad7" (UID: "db854180-8b62-48e3-ad7d-396518053ad7"). InnerVolumeSpecName "kube-api-access-nxmp9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:21:13.749415 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.749397 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db854180-8b62-48e3-ad7d-396518053ad7-istio-token" (OuterVolumeSpecName: "istio-token") pod "db854180-8b62-48e3-ad7d-396518053ad7" (UID: "db854180-8b62-48e3-ad7d-396518053ad7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:21:13.750554 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.750535 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/db854180-8b62-48e3-ad7d-396518053ad7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "db854180-8b62-48e3-ad7d-396518053ad7" (UID: "db854180-8b62-48e3-ad7d-396518053ad7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:21:13.847999 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.847964 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-istio-data\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:13.847999 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.847993 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-istio-envoy\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:13.847999 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.848003 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db854180-8b62-48e3-ad7d-396518053ad7-istio-podinfo\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:13.848274 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.848013 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db854180-8b62-48e3-ad7d-396518053ad7-istio-token\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:13.848274 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.848022 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxmp9\" (UniqueName: \"kubernetes.io/projected/db854180-8b62-48e3-ad7d-396518053ad7-kube-api-access-nxmp9\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:13.848274 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.848031 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db854180-8b62-48e3-ad7d-396518053ad7-workload-certs\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:13.997756 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.997676 2581 generic.go:356] "Generic (PLEG): container finished" podID="db854180-8b62-48e3-ad7d-396518053ad7" containerID="03962e01b2b8f92de920ffbd8cd21ac073dc0223bab00209c59d8eebfabbee7d" exitCode=0 Jul 06 20:21:13.997756 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.997720 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" Jul 06 20:21:13.997935 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.997752 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" event={"ID":"db854180-8b62-48e3-ad7d-396518053ad7","Type":"ContainerDied","Data":"03962e01b2b8f92de920ffbd8cd21ac073dc0223bab00209c59d8eebfabbee7d"} Jul 06 20:21:13.997935 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.997802 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649" event={"ID":"db854180-8b62-48e3-ad7d-396518053ad7","Type":"ContainerDied","Data":"3c1c2ba08d9d131f8620ca4cde5ce9ca0c816afc298a016a52c9066c05af18b3"} Jul 06 20:21:13.997935 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:13.997823 2581 scope.go:117] "RemoveContainer" containerID="03962e01b2b8f92de920ffbd8cd21ac073dc0223bab00209c59d8eebfabbee7d" Jul 06 20:21:14.007395 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:14.007380 2581 scope.go:117] "RemoveContainer" containerID="453b04fcd47b47acb386089c597246a6ef1c466c984592b7c81ee3384b219f40" Jul 06 20:21:14.015168 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:14.015149 2581 scope.go:117] "RemoveContainer" containerID="03962e01b2b8f92de920ffbd8cd21ac073dc0223bab00209c59d8eebfabbee7d" Jul 06 20:21:14.015434 ip-10-0-130-172 kubenswrapper[2581]: E0706 20:21:14.015415 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03962e01b2b8f92de920ffbd8cd21ac073dc0223bab00209c59d8eebfabbee7d\": container with ID starting with 03962e01b2b8f92de920ffbd8cd21ac073dc0223bab00209c59d8eebfabbee7d not found: ID does not exist" containerID="03962e01b2b8f92de920ffbd8cd21ac073dc0223bab00209c59d8eebfabbee7d" Jul 06 20:21:14.015513 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:14.015440 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03962e01b2b8f92de920ffbd8cd21ac073dc0223bab00209c59d8eebfabbee7d"} err="failed to get container status \"03962e01b2b8f92de920ffbd8cd21ac073dc0223bab00209c59d8eebfabbee7d\": rpc error: code = NotFound desc = could not find container \"03962e01b2b8f92de920ffbd8cd21ac073dc0223bab00209c59d8eebfabbee7d\": container with ID starting with 03962e01b2b8f92de920ffbd8cd21ac073dc0223bab00209c59d8eebfabbee7d not found: ID does not exist" Jul 06 20:21:14.015513 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:14.015457 2581 scope.go:117] "RemoveContainer" containerID="453b04fcd47b47acb386089c597246a6ef1c466c984592b7c81ee3384b219f40" Jul 06 20:21:14.015677 ip-10-0-130-172 kubenswrapper[2581]: E0706 20:21:14.015660 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"453b04fcd47b47acb386089c597246a6ef1c466c984592b7c81ee3384b219f40\": container with ID starting with 453b04fcd47b47acb386089c597246a6ef1c466c984592b7c81ee3384b219f40 not found: ID does not exist" containerID="453b04fcd47b47acb386089c597246a6ef1c466c984592b7c81ee3384b219f40" Jul 06 20:21:14.015723 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:14.015680 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"453b04fcd47b47acb386089c597246a6ef1c466c984592b7c81ee3384b219f40"} err="failed to get container status \"453b04fcd47b47acb386089c597246a6ef1c466c984592b7c81ee3384b219f40\": rpc error: code = NotFound desc = could not find container \"453b04fcd47b47acb386089c597246a6ef1c466c984592b7c81ee3384b219f40\": container with ID starting with 453b04fcd47b47acb386089c597246a6ef1c466c984592b7c81ee3384b219f40 not found: ID does not exist" Jul 06 20:21:14.019408 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:14.019387 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649"] Jul 06 20:21:14.022661 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:14.022642 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-e7dc01-openshift-default-746f64f5df-r2649"] Jul 06 20:21:15.063601 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:15.063560 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4"] Jul 06 20:21:15.064046 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:15.063892 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" podUID="caaa1ed6-661e-4dd7-9e9b-151af3aba580" containerName="istio-proxy" containerID="cri-o://d4a03277db70e6707cdb1ae41a33985d0bd0f08671a7feaf7142053a50036241" gracePeriod=30 Jul 06 20:21:15.094957 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:15.094923 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db854180-8b62-48e3-ad7d-396518053ad7" path="/var/lib/kubelet/pods/db854180-8b62-48e3-ad7d-396518053ad7/volumes" Jul 06 20:21:15.924285 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:15.924255 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc"] Jul 06 20:21:15.924718 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:15.924703 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db854180-8b62-48e3-ad7d-396518053ad7" containerName="istio-proxy" Jul 06 20:21:15.924764 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:15.924721 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="db854180-8b62-48e3-ad7d-396518053ad7" containerName="istio-proxy" Jul 06 20:21:15.924824 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:15.924816 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="db854180-8b62-48e3-ad7d-396518053ad7" containerName="istio-proxy" Jul 06 20:21:15.924861 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:15.924826 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="db854180-8b62-48e3-ad7d-396518053ad7" containerName="istio-proxy" Jul 06 20:21:15.929598 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:15.929583 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:15.932173 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:15.932156 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-1a31be-openshift-default-dockercfg-thvf7\"" Jul 06 20:21:15.941904 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:15.941872 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc"] Jul 06 20:21:16.070873 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.070836 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngp4c\" (UniqueName: \"kubernetes.io/projected/dc663053-9e29-4b06-901b-f851cb3acd58-kube-api-access-ngp4c\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.071275 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.070894 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-workload-socket\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.071275 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.070975 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc663053-9e29-4b06-901b-f851cb3acd58-istiod-ca-cert\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.071275 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.071009 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-credential-socket\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.071275 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.071043 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-istio-envoy\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.071275 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.071118 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-istio-data\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.071275 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.071146 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-workload-certs\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.071275 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.071164 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc663053-9e29-4b06-901b-f851cb3acd58-istio-token\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.071275 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.071206 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc663053-9e29-4b06-901b-f851cb3acd58-istio-podinfo\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.172218 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.172179 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc663053-9e29-4b06-901b-f851cb3acd58-istiod-ca-cert\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.172459 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.172240 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-credential-socket\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.172459 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.172375 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-istio-envoy\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.172459 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.172451 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-istio-data\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.172646 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.172473 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-workload-certs\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.172646 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.172489 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc663053-9e29-4b06-901b-f851cb3acd58-istio-token\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.172646 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.172563 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc663053-9e29-4b06-901b-f851cb3acd58-istio-podinfo\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.172803 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.172673 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-credential-socket\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.172803 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.172681 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngp4c\" (UniqueName: \"kubernetes.io/projected/dc663053-9e29-4b06-901b-f851cb3acd58-kube-api-access-ngp4c\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.172803 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.172792 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-workload-socket\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.173510 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.173479 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-istio-data\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.173644 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.173604 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-workload-certs\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.173713 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.173661 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-workload-socket\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.173805 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.173786 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc663053-9e29-4b06-901b-f851cb3acd58-istiod-ca-cert\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.174956 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.174903 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc663053-9e29-4b06-901b-f851cb3acd58-istio-podinfo\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.174956 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.174919 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-istio-envoy\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.188490 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.188455 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc663053-9e29-4b06-901b-f851cb3acd58-istio-token\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.188656 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.188634 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngp4c\" (UniqueName: \"kubernetes.io/projected/dc663053-9e29-4b06-901b-f851cb3acd58-kube-api-access-ngp4c\") pod \"e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.240435 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.240403 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:16.390432 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.390401 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc"] Jul 06 20:21:16.395041 ip-10-0-130-172 kubenswrapper[2581]: W0706 20:21:16.392948 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc663053_9e29_4b06_901b_f851cb3acd58.slice/crio-9a6d62905d409d9014d6a4093bc6ab94b6afe9ee81e22f27310dd56119aff1e7 WatchSource:0}: Error finding container 9a6d62905d409d9014d6a4093bc6ab94b6afe9ee81e22f27310dd56119aff1e7: Status 404 returned error can't find the container with id 9a6d62905d409d9014d6a4093bc6ab94b6afe9ee81e22f27310dd56119aff1e7 Jul 06 20:21:16.395938 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.395904 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:21:16.396042 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.395992 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:21:16.396042 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:16.396029 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:21:17.015728 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:17.015692 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" event={"ID":"dc663053-9e29-4b06-901b-f851cb3acd58","Type":"ContainerStarted","Data":"0c951ff0c85871b64b51dad664b2f148c71c0abeb7efa966f70cec83bfd7c107"} Jul 06 20:21:17.015944 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:17.015746 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" event={"ID":"dc663053-9e29-4b06-901b-f851cb3acd58","Type":"ContainerStarted","Data":"9a6d62905d409d9014d6a4093bc6ab94b6afe9ee81e22f27310dd56119aff1e7"} Jul 06 20:21:17.055764 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:17.055714 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" podStartSLOduration=2.05569841 podStartE2EDuration="2.05569841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:21:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:21:17.05323086 +0000 UTC m=+1976.500467579" watchObservedRunningTime="2026-07-06 20:21:17.05569841 +0000 UTC m=+1976.502935128" Jul 06 20:21:17.241266 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:17.241233 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:17.247028 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:17.247006 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:18.022111 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:18.022076 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:18.023168 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:18.023150 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:20.311663 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.311638 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:21:20.416928 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.416841 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-data\") pod \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " Jul 06 20:21:20.416928 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.416886 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istiod-ca-cert\") pod \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " Jul 06 20:21:20.417133 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.416934 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-workload-certs\") pod \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " Jul 06 20:21:20.417133 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.416968 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzdl2\" (UniqueName: \"kubernetes.io/projected/caaa1ed6-661e-4dd7-9e9b-151af3aba580-kube-api-access-qzdl2\") pod \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " Jul 06 20:21:20.417133 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.417016 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-workload-socket\") pod \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " Jul 06 20:21:20.417133 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.417055 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-token\") pod \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " Jul 06 20:21:20.417133 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.417084 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-podinfo\") pod \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " Jul 06 20:21:20.417409 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.417182 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-envoy\") pod \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " Jul 06 20:21:20.417409 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.417223 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-credential-socket\") pod \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\" (UID: \"caaa1ed6-661e-4dd7-9e9b-151af3aba580\") " Jul 06 20:21:20.417409 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.417236 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "caaa1ed6-661e-4dd7-9e9b-151af3aba580" (UID: "caaa1ed6-661e-4dd7-9e9b-151af3aba580"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:21:20.417409 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.417397 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "caaa1ed6-661e-4dd7-9e9b-151af3aba580" (UID: "caaa1ed6-661e-4dd7-9e9b-151af3aba580"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:21:20.417629 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.417433 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "caaa1ed6-661e-4dd7-9e9b-151af3aba580" (UID: "caaa1ed6-661e-4dd7-9e9b-151af3aba580"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:21:20.417629 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.417611 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "caaa1ed6-661e-4dd7-9e9b-151af3aba580" (UID: "caaa1ed6-661e-4dd7-9e9b-151af3aba580"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:21:20.417731 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.417630 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-workload-certs\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:20.417731 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.417646 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-workload-socket\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:20.418140 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.418110 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-data" (OuterVolumeSpecName: "istio-data") pod "caaa1ed6-661e-4dd7-9e9b-151af3aba580" (UID: "caaa1ed6-661e-4dd7-9e9b-151af3aba580"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:21:20.419461 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.419438 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "caaa1ed6-661e-4dd7-9e9b-151af3aba580" (UID: "caaa1ed6-661e-4dd7-9e9b-151af3aba580"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:21:20.419572 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.419440 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "caaa1ed6-661e-4dd7-9e9b-151af3aba580" (UID: "caaa1ed6-661e-4dd7-9e9b-151af3aba580"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:21:20.419621 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.419584 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caaa1ed6-661e-4dd7-9e9b-151af3aba580-kube-api-access-qzdl2" (OuterVolumeSpecName: "kube-api-access-qzdl2") pod "caaa1ed6-661e-4dd7-9e9b-151af3aba580" (UID: "caaa1ed6-661e-4dd7-9e9b-151af3aba580"). InnerVolumeSpecName "kube-api-access-qzdl2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:21:20.419661 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.419620 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-token" (OuterVolumeSpecName: "istio-token") pod "caaa1ed6-661e-4dd7-9e9b-151af3aba580" (UID: "caaa1ed6-661e-4dd7-9e9b-151af3aba580"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:21:20.518237 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.518202 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-data\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:20.518237 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.518231 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istiod-ca-cert\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:20.518237 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.518246 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzdl2\" (UniqueName: \"kubernetes.io/projected/caaa1ed6-661e-4dd7-9e9b-151af3aba580-kube-api-access-qzdl2\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:20.518510 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.518255 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-token\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:20.518510 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.518263 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-podinfo\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:20.518510 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.518271 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-istio-envoy\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:20.518510 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:20.518280 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caaa1ed6-661e-4dd7-9e9b-151af3aba580-credential-socket\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:21.038285 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:21.038247 2581 generic.go:356] "Generic (PLEG): container finished" podID="caaa1ed6-661e-4dd7-9e9b-151af3aba580" containerID="d4a03277db70e6707cdb1ae41a33985d0bd0f08671a7feaf7142053a50036241" exitCode=0 Jul 06 20:21:21.038285 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:21.038289 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" Jul 06 20:21:21.038543 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:21.038337 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" event={"ID":"caaa1ed6-661e-4dd7-9e9b-151af3aba580","Type":"ContainerDied","Data":"d4a03277db70e6707cdb1ae41a33985d0bd0f08671a7feaf7142053a50036241"} Jul 06 20:21:21.038543 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:21.038398 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4" event={"ID":"caaa1ed6-661e-4dd7-9e9b-151af3aba580","Type":"ContainerDied","Data":"0a70bbd6ec6c6bdd125efbcc22e29601f34043510ad37b9431496d9df19862f6"} Jul 06 20:21:21.038543 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:21.038421 2581 scope.go:117] "RemoveContainer" containerID="d4a03277db70e6707cdb1ae41a33985d0bd0f08671a7feaf7142053a50036241" Jul 06 20:21:21.047762 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:21.047744 2581 scope.go:117] "RemoveContainer" containerID="d7b4f90b42b73637f5698d5d175427309adcda4643e0a8526d559bb2e767c2c5" Jul 06 20:21:21.056115 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:21.056095 2581 scope.go:117] "RemoveContainer" containerID="d4a03277db70e6707cdb1ae41a33985d0bd0f08671a7feaf7142053a50036241" Jul 06 20:21:21.056385 ip-10-0-130-172 kubenswrapper[2581]: E0706 20:21:21.056363 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4a03277db70e6707cdb1ae41a33985d0bd0f08671a7feaf7142053a50036241\": container with ID starting with d4a03277db70e6707cdb1ae41a33985d0bd0f08671a7feaf7142053a50036241 not found: ID does not exist" containerID="d4a03277db70e6707cdb1ae41a33985d0bd0f08671a7feaf7142053a50036241" Jul 06 20:21:21.056468 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:21.056397 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4a03277db70e6707cdb1ae41a33985d0bd0f08671a7feaf7142053a50036241"} err="failed to get container status \"d4a03277db70e6707cdb1ae41a33985d0bd0f08671a7feaf7142053a50036241\": rpc error: code = NotFound desc = could not find container \"d4a03277db70e6707cdb1ae41a33985d0bd0f08671a7feaf7142053a50036241\": container with ID starting with d4a03277db70e6707cdb1ae41a33985d0bd0f08671a7feaf7142053a50036241 not found: ID does not exist" Jul 06 20:21:21.056468 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:21.056420 2581 scope.go:117] "RemoveContainer" containerID="d7b4f90b42b73637f5698d5d175427309adcda4643e0a8526d559bb2e767c2c5" Jul 06 20:21:21.056709 ip-10-0-130-172 kubenswrapper[2581]: E0706 20:21:21.056687 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7b4f90b42b73637f5698d5d175427309adcda4643e0a8526d559bb2e767c2c5\": container with ID starting with d7b4f90b42b73637f5698d5d175427309adcda4643e0a8526d559bb2e767c2c5 not found: ID does not exist" containerID="d7b4f90b42b73637f5698d5d175427309adcda4643e0a8526d559bb2e767c2c5" Jul 06 20:21:21.056759 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:21.056718 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7b4f90b42b73637f5698d5d175427309adcda4643e0a8526d559bb2e767c2c5"} err="failed to get container status \"d7b4f90b42b73637f5698d5d175427309adcda4643e0a8526d559bb2e767c2c5\": rpc error: code = NotFound desc = could not find container \"d7b4f90b42b73637f5698d5d175427309adcda4643e0a8526d559bb2e767c2c5\": container with ID starting with d7b4f90b42b73637f5698d5d175427309adcda4643e0a8526d559bb2e767c2c5 not found: ID does not exist" Jul 06 20:21:21.060178 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:21.060155 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4"] Jul 06 20:21:21.063694 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:21.063676 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ec5878-openshift-default-6cdb5847d7-wcxv4"] Jul 06 20:21:21.096508 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:21.096479 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="caaa1ed6-661e-4dd7-9e9b-151af3aba580" path="/var/lib/kubelet/pods/caaa1ed6-661e-4dd7-9e9b-151af3aba580/volumes" Jul 06 20:21:24.182986 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.182949 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz"] Jul 06 20:21:24.183472 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.183449 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caaa1ed6-661e-4dd7-9e9b-151af3aba580" containerName="istio-proxy" Jul 06 20:21:24.183472 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.183473 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="caaa1ed6-661e-4dd7-9e9b-151af3aba580" containerName="istio-proxy" Jul 06 20:21:24.183703 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.183493 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db854180-8b62-48e3-ad7d-396518053ad7" containerName="istio-proxy" Jul 06 20:21:24.183703 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.183502 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="db854180-8b62-48e3-ad7d-396518053ad7" containerName="istio-proxy" Jul 06 20:21:24.183703 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.183635 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="caaa1ed6-661e-4dd7-9e9b-151af3aba580" containerName="istio-proxy" Jul 06 20:21:24.183703 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.183650 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="caaa1ed6-661e-4dd7-9e9b-151af3aba580" containerName="istio-proxy" Jul 06 20:21:24.188588 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.188569 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.191063 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.191046 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-6c248c-openshift-default-dockercfg-467mf\"" Jul 06 20:21:24.199010 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.198986 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz"] Jul 06 20:21:24.354477 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.354438 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-workload-socket\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.354672 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.354481 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54255\" (UniqueName: \"kubernetes.io/projected/2a282e56-d009-4cac-85ad-7536563a486e-kube-api-access-54255\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.354672 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.354518 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-istio-data\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.354672 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.354617 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a282e56-d009-4cac-85ad-7536563a486e-istiod-ca-cert\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.354672 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.354652 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-workload-certs\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.354923 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.354725 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a282e56-d009-4cac-85ad-7536563a486e-istio-podinfo\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.354923 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.354789 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a282e56-d009-4cac-85ad-7536563a486e-istio-token\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.354923 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.354817 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-istio-envoy\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.354923 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.354844 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-credential-socket\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.455644 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.455557 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a282e56-d009-4cac-85ad-7536563a486e-istio-podinfo\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.455644 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.455614 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a282e56-d009-4cac-85ad-7536563a486e-istio-token\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.455644 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.455634 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-istio-envoy\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.455899 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.455653 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-credential-socket\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.455899 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.455702 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-workload-socket\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.455899 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.455735 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54255\" (UniqueName: \"kubernetes.io/projected/2a282e56-d009-4cac-85ad-7536563a486e-kube-api-access-54255\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.455899 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.455820 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-istio-data\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.455899 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.455879 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a282e56-d009-4cac-85ad-7536563a486e-istiod-ca-cert\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.456166 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.455906 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-workload-certs\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.456222 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.456177 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-credential-socket\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.456281 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.456217 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-workload-socket\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.456281 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.456264 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-istio-data\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.456411 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.456393 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-workload-certs\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.456726 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.456701 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a282e56-d009-4cac-85ad-7536563a486e-istiod-ca-cert\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.458084 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.458061 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a282e56-d009-4cac-85ad-7536563a486e-istio-podinfo\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.458181 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.458092 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-istio-envoy\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.469865 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.469841 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a282e56-d009-4cac-85ad-7536563a486e-istio-token\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.470124 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.470102 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54255\" (UniqueName: \"kubernetes.io/projected/2a282e56-d009-4cac-85ad-7536563a486e-kube-api-access-54255\") pod \"e2e-api-b-6c248c-openshift-default-656dbcb685-945cz\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.499958 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.499928 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:24.641623 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.641599 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz"] Jul 06 20:21:24.643398 ip-10-0-130-172 kubenswrapper[2581]: W0706 20:21:24.643372 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a282e56_d009_4cac_85ad_7536563a486e.slice/crio-0f0de8002dc506c4752305ab93ee5895b8037f54235c909530ede1c542ce4a57 WatchSource:0}: Error finding container 0f0de8002dc506c4752305ab93ee5895b8037f54235c909530ede1c542ce4a57: Status 404 returned error can't find the container with id 0f0de8002dc506c4752305ab93ee5895b8037f54235c909530ede1c542ce4a57 Jul 06 20:21:24.645660 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.645626 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:21:24.645739 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.645690 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:21:24.645739 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:24.645724 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:21:25.070849 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:25.070812 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" event={"ID":"2a282e56-d009-4cac-85ad-7536563a486e","Type":"ContainerStarted","Data":"b472ffaf01cf3715aefa4b61425a3f23b3a56abbac2936f11be15d5c1c1043cf"} Jul 06 20:21:25.071062 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:25.070862 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" event={"ID":"2a282e56-d009-4cac-85ad-7536563a486e","Type":"ContainerStarted","Data":"0f0de8002dc506c4752305ab93ee5895b8037f54235c909530ede1c542ce4a57"} Jul 06 20:21:25.093856 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:25.093810 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" podStartSLOduration=1.093795353 podStartE2EDuration="1.093795353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:21:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:21:25.090924451 +0000 UTC m=+1984.538161170" watchObservedRunningTime="2026-07-06 20:21:25.093795353 +0000 UTC m=+1984.541032072" Jul 06 20:21:25.500729 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:25.500643 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:25.505555 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:25.505529 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:26.076660 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:26.076623 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:26.077474 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:26.077456 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:34.064461 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:34.064422 2581 patch_prober.go:28] interesting pod/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 20:21:34.064866 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:34.064517 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" podUID="dc663053-9e29-4b06-901b-f851cb3acd58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 20:21:35.126899 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:35.126871 2581 generic.go:356] "Generic (PLEG): container finished" podID="dc663053-9e29-4b06-901b-f851cb3acd58" containerID="0c951ff0c85871b64b51dad664b2f148c71c0abeb7efa966f70cec83bfd7c107" exitCode=0 Jul 06 20:21:35.127239 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:35.126945 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" event={"ID":"dc663053-9e29-4b06-901b-f851cb3acd58","Type":"ContainerDied","Data":"0c951ff0c85871b64b51dad664b2f148c71c0abeb7efa966f70cec83bfd7c107"} Jul 06 20:21:35.127459 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:35.127444 2581 scope.go:117] "RemoveContainer" containerID="0c951ff0c85871b64b51dad664b2f148c71c0abeb7efa966f70cec83bfd7c107" Jul 06 20:21:35.128467 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:35.128441 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:21:35.128563 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:35.128505 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:21:35.128563 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:35.128535 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:21:36.134266 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:36.134229 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" event={"ID":"dc663053-9e29-4b06-901b-f851cb3acd58","Type":"ContainerStarted","Data":"c1bd882d2dfdcc34bfa19158b547ffda3f1e3ba31129256b21c6619d5fb2752a"} Jul 06 20:21:36.241254 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:36.241227 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:36.241838 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:36.241524 2581 patch_prober.go:28] interesting pod/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 06 20:21:36.241838 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:36.241576 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" podUID="dc663053-9e29-4b06-901b-f851cb3acd58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 06 20:21:37.241866 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:37.241762 2581 patch_prober.go:28] interesting pod/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 06 20:21:37.242487 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:37.242447 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" podUID="dc663053-9e29-4b06-901b-f851cb3acd58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 06 20:21:38.241560 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:38.241517 2581 patch_prober.go:28] interesting pod/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 06 20:21:38.241774 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:38.241586 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" podUID="dc663053-9e29-4b06-901b-f851cb3acd58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 06 20:21:39.240877 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:39.240842 2581 patch_prober.go:28] interesting pod/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 06 20:21:39.241287 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:39.240891 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" podUID="dc663053-9e29-4b06-901b-f851cb3acd58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 06 20:21:40.248776 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:40.248733 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:40.249923 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:40.249898 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:40.250848 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:40.250826 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:42.119522 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:42.119476 2581 patch_prober.go:28] interesting pod/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 20:21:42.119905 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:42.119545 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" podUID="2a282e56-d009-4cac-85ad-7536563a486e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 20:21:44.187597 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:44.187511 2581 generic.go:356] "Generic (PLEG): container finished" podID="2a282e56-d009-4cac-85ad-7536563a486e" containerID="b472ffaf01cf3715aefa4b61425a3f23b3a56abbac2936f11be15d5c1c1043cf" exitCode=0 Jul 06 20:21:44.188087 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:44.187586 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" event={"ID":"2a282e56-d009-4cac-85ad-7536563a486e","Type":"ContainerDied","Data":"b472ffaf01cf3715aefa4b61425a3f23b3a56abbac2936f11be15d5c1c1043cf"} Jul 06 20:21:44.188155 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:44.188086 2581 scope.go:117] "RemoveContainer" containerID="b472ffaf01cf3715aefa4b61425a3f23b3a56abbac2936f11be15d5c1c1043cf" Jul 06 20:21:44.189269 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:44.189244 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:21:44.189355 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:44.189310 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:21:44.189400 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:44.189363 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:21:45.195272 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:45.195241 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" event={"ID":"2a282e56-d009-4cac-85ad-7536563a486e","Type":"ContainerStarted","Data":"d60b48e60362a0446082dec000b86e632b6d8218f0f2dfb19e87d44cd58a8241"} Jul 06 20:21:45.500534 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:45.500445 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:45.500693 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:45.500651 2581 patch_prober.go:28] interesting pod/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 06 20:21:45.500742 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:45.500698 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" podUID="2a282e56-d009-4cac-85ad-7536563a486e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 06 20:21:46.232108 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:46.232075 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc"] Jul 06 20:21:46.232511 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:46.232347 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" podUID="dc663053-9e29-4b06-901b-f851cb3acd58" containerName="istio-proxy" containerID="cri-o://c1bd882d2dfdcc34bfa19158b547ffda3f1e3ba31129256b21c6619d5fb2752a" gracePeriod=30 Jul 06 20:21:46.500869 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:46.500785 2581 patch_prober.go:28] interesting pod/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 06 20:21:46.500869 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:46.500836 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" podUID="2a282e56-d009-4cac-85ad-7536563a486e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 06 20:21:47.500901 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:47.500863 2581 patch_prober.go:28] interesting pod/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 06 20:21:47.501340 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:47.500921 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" podUID="2a282e56-d009-4cac-85ad-7536563a486e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 06 20:21:48.500920 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:48.500885 2581 patch_prober.go:28] interesting pod/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 06 20:21:48.501290 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:48.500939 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" podUID="2a282e56-d009-4cac-85ad-7536563a486e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 06 20:21:49.503422 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:49.503391 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:49.503852 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:49.503795 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:49.504522 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:49.504501 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:51.498261 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.498203 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:51.530760 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.530727 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-istio-data\") pod \"dc663053-9e29-4b06-901b-f851cb3acd58\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " Jul 06 20:21:51.530933 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.530768 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc663053-9e29-4b06-901b-f851cb3acd58-istiod-ca-cert\") pod \"dc663053-9e29-4b06-901b-f851cb3acd58\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " Jul 06 20:21:51.530933 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.530795 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-workload-certs\") pod \"dc663053-9e29-4b06-901b-f851cb3acd58\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " Jul 06 20:21:51.530933 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.530865 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-istio-envoy\") pod \"dc663053-9e29-4b06-901b-f851cb3acd58\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " Jul 06 20:21:51.530933 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.530904 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-credential-socket\") pod \"dc663053-9e29-4b06-901b-f851cb3acd58\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " Jul 06 20:21:51.530933 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.530926 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc663053-9e29-4b06-901b-f851cb3acd58-istio-podinfo\") pod \"dc663053-9e29-4b06-901b-f851cb3acd58\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " Jul 06 20:21:51.531196 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.530957 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc663053-9e29-4b06-901b-f851cb3acd58-istio-token\") pod \"dc663053-9e29-4b06-901b-f851cb3acd58\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " Jul 06 20:21:51.531196 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.530992 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngp4c\" (UniqueName: \"kubernetes.io/projected/dc663053-9e29-4b06-901b-f851cb3acd58-kube-api-access-ngp4c\") pod \"dc663053-9e29-4b06-901b-f851cb3acd58\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " Jul 06 20:21:51.531196 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.531056 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-workload-socket\") pod \"dc663053-9e29-4b06-901b-f851cb3acd58\" (UID: \"dc663053-9e29-4b06-901b-f851cb3acd58\") " Jul 06 20:21:51.531196 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.531124 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dc663053-9e29-4b06-901b-f851cb3acd58" (UID: "dc663053-9e29-4b06-901b-f851cb3acd58"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:21:51.531196 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.531176 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc663053-9e29-4b06-901b-f851cb3acd58-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dc663053-9e29-4b06-901b-f851cb3acd58" (UID: "dc663053-9e29-4b06-901b-f851cb3acd58"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:21:51.531463 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.531215 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dc663053-9e29-4b06-901b-f851cb3acd58" (UID: "dc663053-9e29-4b06-901b-f851cb3acd58"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:21:51.531463 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.531448 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc663053-9e29-4b06-901b-f851cb3acd58-istiod-ca-cert\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:51.531463 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.531452 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dc663053-9e29-4b06-901b-f851cb3acd58" (UID: "dc663053-9e29-4b06-901b-f851cb3acd58"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:21:51.531581 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.531468 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-workload-certs\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:51.531581 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.531483 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-credential-socket\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:51.531581 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.531518 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-istio-data" (OuterVolumeSpecName: "istio-data") pod "dc663053-9e29-4b06-901b-f851cb3acd58" (UID: "dc663053-9e29-4b06-901b-f851cb3acd58"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:21:51.533395 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.533369 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc663053-9e29-4b06-901b-f851cb3acd58-istio-token" (OuterVolumeSpecName: "istio-token") pod "dc663053-9e29-4b06-901b-f851cb3acd58" (UID: "dc663053-9e29-4b06-901b-f851cb3acd58"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:21:51.533518 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.533458 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dc663053-9e29-4b06-901b-f851cb3acd58-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dc663053-9e29-4b06-901b-f851cb3acd58" (UID: "dc663053-9e29-4b06-901b-f851cb3acd58"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:21:51.533666 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.533643 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc663053-9e29-4b06-901b-f851cb3acd58-kube-api-access-ngp4c" (OuterVolumeSpecName: "kube-api-access-ngp4c") pod "dc663053-9e29-4b06-901b-f851cb3acd58" (UID: "dc663053-9e29-4b06-901b-f851cb3acd58"). InnerVolumeSpecName "kube-api-access-ngp4c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:21:51.533796 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.533771 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dc663053-9e29-4b06-901b-f851cb3acd58" (UID: "dc663053-9e29-4b06-901b-f851cb3acd58"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:21:51.634415 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.634384 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-istio-envoy\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:51.634415 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.634414 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc663053-9e29-4b06-901b-f851cb3acd58-istio-podinfo\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:51.634668 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.634425 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc663053-9e29-4b06-901b-f851cb3acd58-istio-token\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:51.634668 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.634438 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngp4c\" (UniqueName: \"kubernetes.io/projected/dc663053-9e29-4b06-901b-f851cb3acd58-kube-api-access-ngp4c\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:51.634668 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.634450 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-workload-socket\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:51.634668 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:51.634462 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc663053-9e29-4b06-901b-f851cb3acd58-istio-data\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:52.235013 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:52.234979 2581 generic.go:356] "Generic (PLEG): container finished" podID="dc663053-9e29-4b06-901b-f851cb3acd58" containerID="c1bd882d2dfdcc34bfa19158b547ffda3f1e3ba31129256b21c6619d5fb2752a" exitCode=0 Jul 06 20:21:52.235229 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:52.235027 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" Jul 06 20:21:52.235229 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:52.235069 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" event={"ID":"dc663053-9e29-4b06-901b-f851cb3acd58","Type":"ContainerDied","Data":"c1bd882d2dfdcc34bfa19158b547ffda3f1e3ba31129256b21c6619d5fb2752a"} Jul 06 20:21:52.235229 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:52.235133 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc" event={"ID":"dc663053-9e29-4b06-901b-f851cb3acd58","Type":"ContainerDied","Data":"9a6d62905d409d9014d6a4093bc6ab94b6afe9ee81e22f27310dd56119aff1e7"} Jul 06 20:21:52.235229 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:52.235165 2581 scope.go:117] "RemoveContainer" containerID="c1bd882d2dfdcc34bfa19158b547ffda3f1e3ba31129256b21c6619d5fb2752a" Jul 06 20:21:52.244520 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:52.244502 2581 scope.go:117] "RemoveContainer" containerID="0c951ff0c85871b64b51dad664b2f148c71c0abeb7efa966f70cec83bfd7c107" Jul 06 20:21:52.253181 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:52.253163 2581 scope.go:117] "RemoveContainer" containerID="c1bd882d2dfdcc34bfa19158b547ffda3f1e3ba31129256b21c6619d5fb2752a" Jul 06 20:21:52.253556 ip-10-0-130-172 kubenswrapper[2581]: E0706 20:21:52.253489 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1bd882d2dfdcc34bfa19158b547ffda3f1e3ba31129256b21c6619d5fb2752a\": container with ID starting with c1bd882d2dfdcc34bfa19158b547ffda3f1e3ba31129256b21c6619d5fb2752a not found: ID does not exist" containerID="c1bd882d2dfdcc34bfa19158b547ffda3f1e3ba31129256b21c6619d5fb2752a" Jul 06 20:21:52.253556 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:52.253529 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1bd882d2dfdcc34bfa19158b547ffda3f1e3ba31129256b21c6619d5fb2752a"} err="failed to get container status \"c1bd882d2dfdcc34bfa19158b547ffda3f1e3ba31129256b21c6619d5fb2752a\": rpc error: code = NotFound desc = could not find container \"c1bd882d2dfdcc34bfa19158b547ffda3f1e3ba31129256b21c6619d5fb2752a\": container with ID starting with c1bd882d2dfdcc34bfa19158b547ffda3f1e3ba31129256b21c6619d5fb2752a not found: ID does not exist" Jul 06 20:21:52.253556 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:52.253551 2581 scope.go:117] "RemoveContainer" containerID="0c951ff0c85871b64b51dad664b2f148c71c0abeb7efa966f70cec83bfd7c107" Jul 06 20:21:52.254005 ip-10-0-130-172 kubenswrapper[2581]: E0706 20:21:52.253856 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c951ff0c85871b64b51dad664b2f148c71c0abeb7efa966f70cec83bfd7c107\": container with ID starting with 0c951ff0c85871b64b51dad664b2f148c71c0abeb7efa966f70cec83bfd7c107 not found: ID does not exist" containerID="0c951ff0c85871b64b51dad664b2f148c71c0abeb7efa966f70cec83bfd7c107" Jul 06 20:21:52.254005 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:52.253891 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c951ff0c85871b64b51dad664b2f148c71c0abeb7efa966f70cec83bfd7c107"} err="failed to get container status \"0c951ff0c85871b64b51dad664b2f148c71c0abeb7efa966f70cec83bfd7c107\": rpc error: code = NotFound desc = could not find container \"0c951ff0c85871b64b51dad664b2f148c71c0abeb7efa966f70cec83bfd7c107\": container with ID starting with 0c951ff0c85871b64b51dad664b2f148c71c0abeb7efa966f70cec83bfd7c107 not found: ID does not exist" Jul 06 20:21:52.261336 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:52.257954 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc"] Jul 06 20:21:52.266207 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:52.266180 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-1a31be-openshift-default-677f79f4d5-7zczc"] Jul 06 20:21:53.095615 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:53.095588 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc663053-9e29-4b06-901b-f851cb3acd58" path="/var/lib/kubelet/pods/dc663053-9e29-4b06-901b-f851cb3acd58/volumes" Jul 06 20:21:53.163377 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:53.163344 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz"] Jul 06 20:21:53.163601 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:53.163576 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" podUID="2a282e56-d009-4cac-85ad-7536563a486e" containerName="istio-proxy" containerID="cri-o://d60b48e60362a0446082dec000b86e632b6d8218f0f2dfb19e87d44cd58a8241" gracePeriod=30 Jul 06 20:21:54.080265 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.080235 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf"] Jul 06 20:21:54.080686 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.080673 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc663053-9e29-4b06-901b-f851cb3acd58" containerName="istio-proxy" Jul 06 20:21:54.080729 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.080688 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc663053-9e29-4b06-901b-f851cb3acd58" containerName="istio-proxy" Jul 06 20:21:54.080729 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.080704 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caaa1ed6-661e-4dd7-9e9b-151af3aba580" containerName="istio-proxy" Jul 06 20:21:54.080729 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.080711 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="caaa1ed6-661e-4dd7-9e9b-151af3aba580" containerName="istio-proxy" Jul 06 20:21:54.080823 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.080786 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc663053-9e29-4b06-901b-f851cb3acd58" containerName="istio-proxy" Jul 06 20:21:54.080823 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.080793 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc663053-9e29-4b06-901b-f851cb3acd58" containerName="istio-proxy" Jul 06 20:21:54.085512 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.085496 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.088101 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.088083 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-c46beb-openshift-default-dockercfg-jnjv7\"" Jul 06 20:21:54.095674 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.095650 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf"] Jul 06 20:21:54.160546 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.160504 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-podinfo\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.160750 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.160615 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-workload-certs\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.160829 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.160763 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-workload-socket\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.160829 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.160818 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-credential-socket\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.160937 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.160852 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istiod-ca-cert\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.160937 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.160888 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-data\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.161043 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.160968 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttbs4\" (UniqueName: \"kubernetes.io/projected/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-kube-api-access-ttbs4\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.161043 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.161019 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-envoy\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.161159 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.161135 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-token\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.262249 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.262212 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-workload-certs\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.262249 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.262250 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-workload-socket\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.262507 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.262273 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-credential-socket\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.262507 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.262293 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istiod-ca-cert\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.262507 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.262311 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-data\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.262507 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.262380 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttbs4\" (UniqueName: \"kubernetes.io/projected/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-kube-api-access-ttbs4\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.262507 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.262409 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-envoy\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.262770 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.262555 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-token\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.262770 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.262632 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-podinfo\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.262770 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.262661 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-workload-certs\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.262770 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.262712 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-credential-socket\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.262942 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.262812 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-data\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.262985 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.262946 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-workload-socket\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.263069 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.263046 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istiod-ca-cert\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.264583 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.264565 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-envoy\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.265084 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.265063 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-podinfo\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.271165 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.271142 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-token\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.271415 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.271396 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttbs4\" (UniqueName: \"kubernetes.io/projected/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-kube-api-access-ttbs4\") pod \"e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.397600 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.397501 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:54.546891 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.546864 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf"] Jul 06 20:21:54.548903 ip-10-0-130-172 kubenswrapper[2581]: W0706 20:21:54.548875 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6a9c990_9d8c_4879_8ce5_20cfce81f7e9.slice/crio-506d122c87bb948a5ddd23c3cd3ea7866bf11e94615ece11b7387cc525b4eacc WatchSource:0}: Error finding container 506d122c87bb948a5ddd23c3cd3ea7866bf11e94615ece11b7387cc525b4eacc: Status 404 returned error can't find the container with id 506d122c87bb948a5ddd23c3cd3ea7866bf11e94615ece11b7387cc525b4eacc Jul 06 20:21:54.551176 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.551135 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:21:54.551259 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.551223 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:21:54.551299 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:54.551263 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:21:55.255670 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:55.255635 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" event={"ID":"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9","Type":"ContainerStarted","Data":"4e4bbd57790f754272d79bd0a6fcfd662b89c00e33413e59e2a93a093d2e7223"} Jul 06 20:21:55.256114 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:55.255695 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" event={"ID":"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9","Type":"ContainerStarted","Data":"506d122c87bb948a5ddd23c3cd3ea7866bf11e94615ece11b7387cc525b4eacc"} Jul 06 20:21:55.280341 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:55.280272 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" podStartSLOduration=1.280255414 podStartE2EDuration="1.280255414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:21:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:21:55.279703796 +0000 UTC m=+2014.726940515" watchObservedRunningTime="2026-07-06 20:21:55.280255414 +0000 UTC m=+2014.727492131" Jul 06 20:21:55.398097 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:55.398067 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:55.403140 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:55.403117 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:56.261380 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:56.261346 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:56.262286 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:56.262266 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:21:58.446077 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.446052 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:58.505495 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.505466 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54255\" (UniqueName: \"kubernetes.io/projected/2a282e56-d009-4cac-85ad-7536563a486e-kube-api-access-54255\") pod \"2a282e56-d009-4cac-85ad-7536563a486e\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " Jul 06 20:21:58.505693 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.505526 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-workload-certs\") pod \"2a282e56-d009-4cac-85ad-7536563a486e\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " Jul 06 20:21:58.505693 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.505563 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a282e56-d009-4cac-85ad-7536563a486e-istiod-ca-cert\") pod \"2a282e56-d009-4cac-85ad-7536563a486e\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " Jul 06 20:21:58.505693 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.505595 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-istio-envoy\") pod \"2a282e56-d009-4cac-85ad-7536563a486e\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " Jul 06 20:21:58.505693 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.505626 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a282e56-d009-4cac-85ad-7536563a486e-istio-token\") pod \"2a282e56-d009-4cac-85ad-7536563a486e\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " Jul 06 20:21:58.505693 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.505674 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-workload-socket\") pod \"2a282e56-d009-4cac-85ad-7536563a486e\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " Jul 06 20:21:58.505954 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.505704 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a282e56-d009-4cac-85ad-7536563a486e-istio-podinfo\") pod \"2a282e56-d009-4cac-85ad-7536563a486e\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " Jul 06 20:21:58.505954 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.505764 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-credential-socket\") pod \"2a282e56-d009-4cac-85ad-7536563a486e\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " Jul 06 20:21:58.505954 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.505824 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-istio-data\") pod \"2a282e56-d009-4cac-85ad-7536563a486e\" (UID: \"2a282e56-d009-4cac-85ad-7536563a486e\") " Jul 06 20:21:58.505954 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.505894 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2a282e56-d009-4cac-85ad-7536563a486e" (UID: "2a282e56-d009-4cac-85ad-7536563a486e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:21:58.506573 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.505958 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2a282e56-d009-4cac-85ad-7536563a486e" (UID: "2a282e56-d009-4cac-85ad-7536563a486e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:21:58.506573 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.506006 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a282e56-d009-4cac-85ad-7536563a486e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2a282e56-d009-4cac-85ad-7536563a486e" (UID: "2a282e56-d009-4cac-85ad-7536563a486e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:21:58.506573 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.506256 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2a282e56-d009-4cac-85ad-7536563a486e" (UID: "2a282e56-d009-4cac-85ad-7536563a486e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:21:58.506573 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.506382 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-workload-certs\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:58.506573 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.506403 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a282e56-d009-4cac-85ad-7536563a486e-istiod-ca-cert\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:58.506573 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.506417 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-workload-socket\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:58.506573 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.506430 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-credential-socket\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:58.506915 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.506849 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-istio-data" (OuterVolumeSpecName: "istio-data") pod "2a282e56-d009-4cac-85ad-7536563a486e" (UID: "2a282e56-d009-4cac-85ad-7536563a486e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:21:58.507894 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.507866 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a282e56-d009-4cac-85ad-7536563a486e-istio-token" (OuterVolumeSpecName: "istio-token") pod "2a282e56-d009-4cac-85ad-7536563a486e" (UID: "2a282e56-d009-4cac-85ad-7536563a486e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:21:58.508013 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.507961 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a282e56-d009-4cac-85ad-7536563a486e-kube-api-access-54255" (OuterVolumeSpecName: "kube-api-access-54255") pod "2a282e56-d009-4cac-85ad-7536563a486e" (UID: "2a282e56-d009-4cac-85ad-7536563a486e"). InnerVolumeSpecName "kube-api-access-54255". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:21:58.508013 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.507968 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2a282e56-d009-4cac-85ad-7536563a486e" (UID: "2a282e56-d009-4cac-85ad-7536563a486e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:21:58.508236 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.508221 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2a282e56-d009-4cac-85ad-7536563a486e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2a282e56-d009-4cac-85ad-7536563a486e" (UID: "2a282e56-d009-4cac-85ad-7536563a486e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:21:58.607156 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.607128 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-istio-envoy\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:58.607156 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.607152 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a282e56-d009-4cac-85ad-7536563a486e-istio-token\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:58.607156 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.607161 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a282e56-d009-4cac-85ad-7536563a486e-istio-podinfo\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:58.607438 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.607169 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a282e56-d009-4cac-85ad-7536563a486e-istio-data\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:58.607438 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:58.607178 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54255\" (UniqueName: \"kubernetes.io/projected/2a282e56-d009-4cac-85ad-7536563a486e-kube-api-access-54255\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:21:59.278981 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:59.278945 2581 generic.go:356] "Generic (PLEG): container finished" podID="2a282e56-d009-4cac-85ad-7536563a486e" containerID="d60b48e60362a0446082dec000b86e632b6d8218f0f2dfb19e87d44cd58a8241" exitCode=0 Jul 06 20:21:59.279149 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:59.278990 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" Jul 06 20:21:59.279149 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:59.278990 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" event={"ID":"2a282e56-d009-4cac-85ad-7536563a486e","Type":"ContainerDied","Data":"d60b48e60362a0446082dec000b86e632b6d8218f0f2dfb19e87d44cd58a8241"} Jul 06 20:21:59.279149 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:59.279112 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz" event={"ID":"2a282e56-d009-4cac-85ad-7536563a486e","Type":"ContainerDied","Data":"0f0de8002dc506c4752305ab93ee5895b8037f54235c909530ede1c542ce4a57"} Jul 06 20:21:59.279149 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:59.279138 2581 scope.go:117] "RemoveContainer" containerID="d60b48e60362a0446082dec000b86e632b6d8218f0f2dfb19e87d44cd58a8241" Jul 06 20:21:59.288876 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:59.288861 2581 scope.go:117] "RemoveContainer" containerID="b472ffaf01cf3715aefa4b61425a3f23b3a56abbac2936f11be15d5c1c1043cf" Jul 06 20:21:59.297489 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:59.297458 2581 scope.go:117] "RemoveContainer" containerID="d60b48e60362a0446082dec000b86e632b6d8218f0f2dfb19e87d44cd58a8241" Jul 06 20:21:59.297733 ip-10-0-130-172 kubenswrapper[2581]: E0706 20:21:59.297713 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d60b48e60362a0446082dec000b86e632b6d8218f0f2dfb19e87d44cd58a8241\": container with ID starting with d60b48e60362a0446082dec000b86e632b6d8218f0f2dfb19e87d44cd58a8241 not found: ID does not exist" containerID="d60b48e60362a0446082dec000b86e632b6d8218f0f2dfb19e87d44cd58a8241" Jul 06 20:21:59.297794 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:59.297739 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d60b48e60362a0446082dec000b86e632b6d8218f0f2dfb19e87d44cd58a8241"} err="failed to get container status \"d60b48e60362a0446082dec000b86e632b6d8218f0f2dfb19e87d44cd58a8241\": rpc error: code = NotFound desc = could not find container \"d60b48e60362a0446082dec000b86e632b6d8218f0f2dfb19e87d44cd58a8241\": container with ID starting with d60b48e60362a0446082dec000b86e632b6d8218f0f2dfb19e87d44cd58a8241 not found: ID does not exist" Jul 06 20:21:59.297794 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:59.297756 2581 scope.go:117] "RemoveContainer" containerID="b472ffaf01cf3715aefa4b61425a3f23b3a56abbac2936f11be15d5c1c1043cf" Jul 06 20:21:59.297986 ip-10-0-130-172 kubenswrapper[2581]: E0706 20:21:59.297972 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b472ffaf01cf3715aefa4b61425a3f23b3a56abbac2936f11be15d5c1c1043cf\": container with ID starting with b472ffaf01cf3715aefa4b61425a3f23b3a56abbac2936f11be15d5c1c1043cf not found: ID does not exist" containerID="b472ffaf01cf3715aefa4b61425a3f23b3a56abbac2936f11be15d5c1c1043cf" Jul 06 20:21:59.298026 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:59.297993 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b472ffaf01cf3715aefa4b61425a3f23b3a56abbac2936f11be15d5c1c1043cf"} err="failed to get container status \"b472ffaf01cf3715aefa4b61425a3f23b3a56abbac2936f11be15d5c1c1043cf\": rpc error: code = NotFound desc = could not find container \"b472ffaf01cf3715aefa4b61425a3f23b3a56abbac2936f11be15d5c1c1043cf\": container with ID starting with b472ffaf01cf3715aefa4b61425a3f23b3a56abbac2936f11be15d5c1c1043cf not found: ID does not exist" Jul 06 20:21:59.298893 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:59.298876 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz"] Jul 06 20:21:59.305351 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:21:59.305334 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-6c248c-openshift-default-656dbcb685-945cz"] Jul 06 20:22:01.096492 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:01.096459 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a282e56-d009-4cac-85ad-7536563a486e" path="/var/lib/kubelet/pods/2a282e56-d009-4cac-85ad-7536563a486e/volumes" Jul 06 20:22:02.298305 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.298273 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r"] Jul 06 20:22:02.298865 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.298828 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a282e56-d009-4cac-85ad-7536563a486e" containerName="istio-proxy" Jul 06 20:22:02.298865 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.298851 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a282e56-d009-4cac-85ad-7536563a486e" containerName="istio-proxy" Jul 06 20:22:02.299016 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.298876 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc663053-9e29-4b06-901b-f851cb3acd58" containerName="istio-proxy" Jul 06 20:22:02.299016 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.298886 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc663053-9e29-4b06-901b-f851cb3acd58" containerName="istio-proxy" Jul 06 20:22:02.299016 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.299009 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a282e56-d009-4cac-85ad-7536563a486e" containerName="istio-proxy" Jul 06 20:22:02.299171 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.299024 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a282e56-d009-4cac-85ad-7536563a486e" containerName="istio-proxy" Jul 06 20:22:02.302779 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.302759 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.305257 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.305236 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-a2776d-openshift-default-dockercfg-mzfxz\"" Jul 06 20:22:02.315717 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.315692 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r"] Jul 06 20:22:02.447115 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.447069 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-workload-socket\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.447115 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.447116 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/621261fa-6849-4fdd-b08a-ba7789963e94-istiod-ca-cert\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.447397 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.447245 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-credential-socket\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.447397 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.447285 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/621261fa-6849-4fdd-b08a-ba7789963e94-istio-podinfo\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.447397 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.447312 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/621261fa-6849-4fdd-b08a-ba7789963e94-istio-token\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.447397 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.447364 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-istio-data\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.447628 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.447463 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-istio-envoy\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.447628 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.447511 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbljg\" (UniqueName: \"kubernetes.io/projected/621261fa-6849-4fdd-b08a-ba7789963e94-kube-api-access-vbljg\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.447628 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.447543 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-workload-certs\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.548908 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.548800 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-istio-envoy\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.548908 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.548852 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbljg\" (UniqueName: \"kubernetes.io/projected/621261fa-6849-4fdd-b08a-ba7789963e94-kube-api-access-vbljg\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.548908 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.548876 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-workload-certs\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.548908 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.548900 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-workload-socket\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.549259 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.548939 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/621261fa-6849-4fdd-b08a-ba7789963e94-istiod-ca-cert\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.549259 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.549015 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-credential-socket\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.549259 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.549040 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/621261fa-6849-4fdd-b08a-ba7789963e94-istio-podinfo\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.549259 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.549062 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/621261fa-6849-4fdd-b08a-ba7789963e94-istio-token\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.549259 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.549091 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-istio-data\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.549550 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.549344 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-workload-certs\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.549550 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.549384 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-workload-socket\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.549550 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.549415 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-credential-socket\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.549713 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.549677 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-istio-data\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.549968 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.549945 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/621261fa-6849-4fdd-b08a-ba7789963e94-istiod-ca-cert\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.551224 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.551202 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-istio-envoy\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.551629 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.551606 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/621261fa-6849-4fdd-b08a-ba7789963e94-istio-podinfo\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.557626 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.557601 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/621261fa-6849-4fdd-b08a-ba7789963e94-istio-token\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.557965 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.557948 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbljg\" (UniqueName: \"kubernetes.io/projected/621261fa-6849-4fdd-b08a-ba7789963e94-kube-api-access-vbljg\") pod \"e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.615065 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.615029 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:02.761674 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.761648 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r"] Jul 06 20:22:02.763181 ip-10-0-130-172 kubenswrapper[2581]: W0706 20:22:02.763156 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod621261fa_6849_4fdd_b08a_ba7789963e94.slice/crio-c2a0c99fd1bb0f7dc557f7262edacf6f990542ff041d17bc4d3e767a203a793c WatchSource:0}: Error finding container c2a0c99fd1bb0f7dc557f7262edacf6f990542ff041d17bc4d3e767a203a793c: Status 404 returned error can't find the container with id c2a0c99fd1bb0f7dc557f7262edacf6f990542ff041d17bc4d3e767a203a793c Jul 06 20:22:02.765180 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.765151 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:22:02.765244 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.765216 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:22:02.765244 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:02.765240 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:22:03.306660 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:03.306623 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" event={"ID":"621261fa-6849-4fdd-b08a-ba7789963e94","Type":"ContainerStarted","Data":"14c7501dcbc56235271e6ff1ed9f7cfccfa53cefa06580a24ce19d1673a0de82"} Jul 06 20:22:03.307076 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:03.306689 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" event={"ID":"621261fa-6849-4fdd-b08a-ba7789963e94","Type":"ContainerStarted","Data":"c2a0c99fd1bb0f7dc557f7262edacf6f990542ff041d17bc4d3e767a203a793c"} Jul 06 20:22:03.329735 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:03.329690 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" podStartSLOduration=1.329674789 podStartE2EDuration="1.329674789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:22:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:22:03.327742449 +0000 UTC m=+2022.774979167" watchObservedRunningTime="2026-07-06 20:22:03.329674789 +0000 UTC m=+2022.776911507" Jul 06 20:22:03.615951 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:03.615876 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:03.620840 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:03.620777 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:04.313464 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:04.313436 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:04.314468 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:04.314451 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:11.263854 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:11.263494 2581 patch_prober.go:28] interesting pod/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 06 20:22:11.263854 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:11.263554 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" podUID="a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 06 20:22:11.358254 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:11.357162 2581 generic.go:356] "Generic (PLEG): container finished" podID="a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" containerID="4e4bbd57790f754272d79bd0a6fcfd662b89c00e33413e59e2a93a093d2e7223" exitCode=0 Jul 06 20:22:11.358254 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:11.357345 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" event={"ID":"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9","Type":"ContainerDied","Data":"4e4bbd57790f754272d79bd0a6fcfd662b89c00e33413e59e2a93a093d2e7223"} Jul 06 20:22:11.358254 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:11.357856 2581 scope.go:117] "RemoveContainer" containerID="4e4bbd57790f754272d79bd0a6fcfd662b89c00e33413e59e2a93a093d2e7223" Jul 06 20:22:11.359127 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:11.359089 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:22:11.359420 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:11.359394 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:22:11.359602 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:11.359581 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:22:12.370441 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:12.370397 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" event={"ID":"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9","Type":"ContainerStarted","Data":"e0bc7db9d9a370fcd4349235b4cf64bc7fff26fcb4a43da7132bc1526429bf01"} Jul 06 20:22:12.398123 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:12.398088 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:22:12.398492 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:12.398449 2581 patch_prober.go:28] interesting pod/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 06 20:22:12.398634 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:12.398527 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" podUID="a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 06 20:22:13.398000 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:13.397965 2581 patch_prober.go:28] interesting pod/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 06 20:22:13.398409 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:13.398014 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" podUID="a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 06 20:22:14.398878 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:14.398845 2581 patch_prober.go:28] interesting pod/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 06 20:22:14.399355 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:14.399308 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" podUID="a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 06 20:22:15.398846 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:15.398807 2581 patch_prober.go:28] interesting pod/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 06 20:22:15.399007 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:15.398872 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" podUID="a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 06 20:22:16.398140 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:16.398095 2581 patch_prober.go:28] interesting pod/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 06 20:22:16.398355 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:16.398173 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" podUID="a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 06 20:22:17.402657 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:17.402625 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:22:17.403129 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:17.403006 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:22:17.404066 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:17.404046 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:22:18.413234 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:18.413199 2581 generic.go:356] "Generic (PLEG): container finished" podID="621261fa-6849-4fdd-b08a-ba7789963e94" containerID="14c7501dcbc56235271e6ff1ed9f7cfccfa53cefa06580a24ce19d1673a0de82" exitCode=0 Jul 06 20:22:18.413658 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:18.413270 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" event={"ID":"621261fa-6849-4fdd-b08a-ba7789963e94","Type":"ContainerDied","Data":"14c7501dcbc56235271e6ff1ed9f7cfccfa53cefa06580a24ce19d1673a0de82"} Jul 06 20:22:18.413786 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:18.413771 2581 scope.go:117] "RemoveContainer" containerID="14c7501dcbc56235271e6ff1ed9f7cfccfa53cefa06580a24ce19d1673a0de82" Jul 06 20:22:18.416019 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:18.415481 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:22:18.416019 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:18.415628 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:22:18.416019 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:18.415725 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 20:22:19.313840 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:19.313803 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:19.420338 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:19.420285 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" event={"ID":"621261fa-6849-4fdd-b08a-ba7789963e94","Type":"ContainerStarted","Data":"6134980c09a4f5426e3c9a487240aa3e76307212c1ab0cefacda46f59cff1042"} Jul 06 20:22:19.616118 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:19.616033 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:19.616975 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:19.616954 2581 patch_prober.go:28] interesting pod/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 06 20:22:19.617101 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:19.616999 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" podUID="621261fa-6849-4fdd-b08a-ba7789963e94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 06 20:22:20.615898 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:20.615855 2581 patch_prober.go:28] interesting pod/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 06 20:22:20.616284 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:20.615934 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" podUID="621261fa-6849-4fdd-b08a-ba7789963e94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 06 20:22:21.615578 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:21.615538 2581 patch_prober.go:28] interesting pod/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 06 20:22:21.615756 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:21.615596 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" podUID="621261fa-6849-4fdd-b08a-ba7789963e94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 06 20:22:22.616158 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:22.616120 2581 patch_prober.go:28] interesting pod/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 06 20:22:22.616621 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:22.616174 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" podUID="621261fa-6849-4fdd-b08a-ba7789963e94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 06 20:22:23.644719 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:23.644681 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:23.645206 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:23.645051 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:22:23.645981 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:22:23.645958 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:23:21.184632 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:23:21.184600 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/2.log" Jul 06 20:23:21.191281 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:23:21.191261 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/2.log" Jul 06 20:28:10.781008 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:10.780974 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf"] Jul 06 20:28:10.781564 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:10.781184 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" podUID="a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" containerName="istio-proxy" containerID="cri-o://e0bc7db9d9a370fcd4349235b4cf64bc7fff26fcb4a43da7132bc1526429bf01" gracePeriod=30 Jul 06 20:28:16.042603 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.042536 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:28:16.128265 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.128229 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-workload-certs\") pod \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " Jul 06 20:28:16.128511 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.128308 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-workload-socket\") pod \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " Jul 06 20:28:16.128511 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.128407 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-credential-socket\") pod \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " Jul 06 20:28:16.128511 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.128438 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istiod-ca-cert\") pod \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " Jul 06 20:28:16.128511 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.128497 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-token\") pod \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " Jul 06 20:28:16.128729 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.128532 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-envoy\") pod \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " Jul 06 20:28:16.128729 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.128531 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" (UID: "a6a9c990-9d8c-4879-8ce5-20cfce81f7e9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:28:16.128729 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.128569 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-podinfo\") pod \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " Jul 06 20:28:16.128729 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.128596 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-data\") pod \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " Jul 06 20:28:16.128729 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.128628 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttbs4\" (UniqueName: \"kubernetes.io/projected/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-kube-api-access-ttbs4\") pod \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\" (UID: \"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9\") " Jul 06 20:28:16.128729 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.128624 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" (UID: "a6a9c990-9d8c-4879-8ce5-20cfce81f7e9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:28:16.128729 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.128718 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" (UID: "a6a9c990-9d8c-4879-8ce5-20cfce81f7e9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:28:16.129104 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.128885 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" (UID: "a6a9c990-9d8c-4879-8ce5-20cfce81f7e9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:28:16.129104 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.129000 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-credential-socket\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:28:16.129104 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.129018 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istiod-ca-cert\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:28:16.129104 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.129032 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-workload-certs\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:28:16.129104 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.129042 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-workload-socket\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:28:16.129442 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.129424 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-data" (OuterVolumeSpecName: "istio-data") pod "a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" (UID: "a6a9c990-9d8c-4879-8ce5-20cfce81f7e9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:28:16.130702 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.130667 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" (UID: "a6a9c990-9d8c-4879-8ce5-20cfce81f7e9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:28:16.130832 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.130785 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-kube-api-access-ttbs4" (OuterVolumeSpecName: "kube-api-access-ttbs4") pod "a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" (UID: "a6a9c990-9d8c-4879-8ce5-20cfce81f7e9"). InnerVolumeSpecName "kube-api-access-ttbs4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:28:16.131085 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.131066 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-token" (OuterVolumeSpecName: "istio-token") pod "a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" (UID: "a6a9c990-9d8c-4879-8ce5-20cfce81f7e9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:28:16.131156 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.131096 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" (UID: "a6a9c990-9d8c-4879-8ce5-20cfce81f7e9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:28:16.229950 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.229916 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-token\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:28:16.229950 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.229941 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-envoy\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:28:16.229950 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.229951 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-podinfo\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:28:16.229950 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.229959 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-istio-data\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:28:16.230204 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.229968 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttbs4\" (UniqueName: \"kubernetes.io/projected/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9-kube-api-access-ttbs4\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:28:16.286797 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.286759 2581 generic.go:356] "Generic (PLEG): container finished" podID="a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" containerID="e0bc7db9d9a370fcd4349235b4cf64bc7fff26fcb4a43da7132bc1526429bf01" exitCode=0 Jul 06 20:28:16.286974 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.286814 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" Jul 06 20:28:16.286974 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.286813 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" event={"ID":"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9","Type":"ContainerDied","Data":"e0bc7db9d9a370fcd4349235b4cf64bc7fff26fcb4a43da7132bc1526429bf01"} Jul 06 20:28:16.286974 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.286956 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf" event={"ID":"a6a9c990-9d8c-4879-8ce5-20cfce81f7e9","Type":"ContainerDied","Data":"506d122c87bb948a5ddd23c3cd3ea7866bf11e94615ece11b7387cc525b4eacc"} Jul 06 20:28:16.287161 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.286999 2581 scope.go:117] "RemoveContainer" containerID="e0bc7db9d9a370fcd4349235b4cf64bc7fff26fcb4a43da7132bc1526429bf01" Jul 06 20:28:16.296131 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.296114 2581 scope.go:117] "RemoveContainer" containerID="4e4bbd57790f754272d79bd0a6fcfd662b89c00e33413e59e2a93a093d2e7223" Jul 06 20:28:16.304174 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.304156 2581 scope.go:117] "RemoveContainer" containerID="e0bc7db9d9a370fcd4349235b4cf64bc7fff26fcb4a43da7132bc1526429bf01" Jul 06 20:28:16.304488 ip-10-0-130-172 kubenswrapper[2581]: E0706 20:28:16.304469 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0bc7db9d9a370fcd4349235b4cf64bc7fff26fcb4a43da7132bc1526429bf01\": container with ID starting with e0bc7db9d9a370fcd4349235b4cf64bc7fff26fcb4a43da7132bc1526429bf01 not found: ID does not exist" containerID="e0bc7db9d9a370fcd4349235b4cf64bc7fff26fcb4a43da7132bc1526429bf01" Jul 06 20:28:16.304555 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.304497 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0bc7db9d9a370fcd4349235b4cf64bc7fff26fcb4a43da7132bc1526429bf01"} err="failed to get container status \"e0bc7db9d9a370fcd4349235b4cf64bc7fff26fcb4a43da7132bc1526429bf01\": rpc error: code = NotFound desc = could not find container \"e0bc7db9d9a370fcd4349235b4cf64bc7fff26fcb4a43da7132bc1526429bf01\": container with ID starting with e0bc7db9d9a370fcd4349235b4cf64bc7fff26fcb4a43da7132bc1526429bf01 not found: ID does not exist" Jul 06 20:28:16.304555 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.304515 2581 scope.go:117] "RemoveContainer" containerID="4e4bbd57790f754272d79bd0a6fcfd662b89c00e33413e59e2a93a093d2e7223" Jul 06 20:28:16.304773 ip-10-0-130-172 kubenswrapper[2581]: E0706 20:28:16.304758 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e4bbd57790f754272d79bd0a6fcfd662b89c00e33413e59e2a93a093d2e7223\": container with ID starting with 4e4bbd57790f754272d79bd0a6fcfd662b89c00e33413e59e2a93a093d2e7223 not found: ID does not exist" containerID="4e4bbd57790f754272d79bd0a6fcfd662b89c00e33413e59e2a93a093d2e7223" Jul 06 20:28:16.304811 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.304779 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e4bbd57790f754272d79bd0a6fcfd662b89c00e33413e59e2a93a093d2e7223"} err="failed to get container status \"4e4bbd57790f754272d79bd0a6fcfd662b89c00e33413e59e2a93a093d2e7223\": rpc error: code = NotFound desc = could not find container \"4e4bbd57790f754272d79bd0a6fcfd662b89c00e33413e59e2a93a093d2e7223\": container with ID starting with 4e4bbd57790f754272d79bd0a6fcfd662b89c00e33413e59e2a93a093d2e7223 not found: ID does not exist" Jul 06 20:28:16.310269 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.310245 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf"] Jul 06 20:28:16.312269 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:16.312247 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-c46beb-openshift-default-9cfff59f5-r8dwf"] Jul 06 20:28:17.100124 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:17.100085 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" path="/var/lib/kubelet/pods/a6a9c990-9d8c-4879-8ce5-20cfce81f7e9/volumes" Jul 06 20:28:17.792796 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:17.792756 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r"] Jul 06 20:28:17.793000 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:17.792977 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" podUID="621261fa-6849-4fdd-b08a-ba7789963e94" containerName="istio-proxy" containerID="cri-o://6134980c09a4f5426e3c9a487240aa3e76307212c1ab0cefacda46f59cff1042" gracePeriod=30 Jul 06 20:28:21.217616 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:21.217589 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/2.log" Jul 06 20:28:21.225367 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:21.225343 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/2.log" Jul 06 20:28:23.062588 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.062529 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:28:23.200214 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.200176 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/621261fa-6849-4fdd-b08a-ba7789963e94-istio-token\") pod \"621261fa-6849-4fdd-b08a-ba7789963e94\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " Jul 06 20:28:23.200427 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.200260 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-workload-socket\") pod \"621261fa-6849-4fdd-b08a-ba7789963e94\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " Jul 06 20:28:23.200427 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.200312 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-istio-envoy\") pod \"621261fa-6849-4fdd-b08a-ba7789963e94\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " Jul 06 20:28:23.200555 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.200438 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbljg\" (UniqueName: \"kubernetes.io/projected/621261fa-6849-4fdd-b08a-ba7789963e94-kube-api-access-vbljg\") pod \"621261fa-6849-4fdd-b08a-ba7789963e94\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " Jul 06 20:28:23.200555 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.200475 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-credential-socket\") pod \"621261fa-6849-4fdd-b08a-ba7789963e94\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " Jul 06 20:28:23.200555 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.200505 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/621261fa-6849-4fdd-b08a-ba7789963e94-istiod-ca-cert\") pod \"621261fa-6849-4fdd-b08a-ba7789963e94\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " Jul 06 20:28:23.200555 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.200532 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-workload-certs\") pod \"621261fa-6849-4fdd-b08a-ba7789963e94\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " Jul 06 20:28:23.200760 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.200575 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-istio-data\") pod \"621261fa-6849-4fdd-b08a-ba7789963e94\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " Jul 06 20:28:23.200760 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.200611 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/621261fa-6849-4fdd-b08a-ba7789963e94-istio-podinfo\") pod \"621261fa-6849-4fdd-b08a-ba7789963e94\" (UID: \"621261fa-6849-4fdd-b08a-ba7789963e94\") " Jul 06 20:28:23.200760 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.200605 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "621261fa-6849-4fdd-b08a-ba7789963e94" (UID: "621261fa-6849-4fdd-b08a-ba7789963e94"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:28:23.200916 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.200749 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "621261fa-6849-4fdd-b08a-ba7789963e94" (UID: "621261fa-6849-4fdd-b08a-ba7789963e94"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:28:23.200916 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.200846 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "621261fa-6849-4fdd-b08a-ba7789963e94" (UID: "621261fa-6849-4fdd-b08a-ba7789963e94"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:28:23.201050 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.201027 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-workload-socket\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:28:23.201110 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.201054 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-credential-socket\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:28:23.201110 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.201071 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-workload-certs\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:28:23.201593 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.201562 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/621261fa-6849-4fdd-b08a-ba7789963e94-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "621261fa-6849-4fdd-b08a-ba7789963e94" (UID: "621261fa-6849-4fdd-b08a-ba7789963e94"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 20:28:23.201704 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.201588 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-istio-data" (OuterVolumeSpecName: "istio-data") pod "621261fa-6849-4fdd-b08a-ba7789963e94" (UID: "621261fa-6849-4fdd-b08a-ba7789963e94"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:28:23.202719 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.202690 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "621261fa-6849-4fdd-b08a-ba7789963e94" (UID: "621261fa-6849-4fdd-b08a-ba7789963e94"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 20:28:23.202820 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.202718 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/621261fa-6849-4fdd-b08a-ba7789963e94-kube-api-access-vbljg" (OuterVolumeSpecName: "kube-api-access-vbljg") pod "621261fa-6849-4fdd-b08a-ba7789963e94" (UID: "621261fa-6849-4fdd-b08a-ba7789963e94"). InnerVolumeSpecName "kube-api-access-vbljg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:28:23.202896 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.202878 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/621261fa-6849-4fdd-b08a-ba7789963e94-istio-token" (OuterVolumeSpecName: "istio-token") pod "621261fa-6849-4fdd-b08a-ba7789963e94" (UID: "621261fa-6849-4fdd-b08a-ba7789963e94"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 20:28:23.202997 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.202981 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/621261fa-6849-4fdd-b08a-ba7789963e94-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "621261fa-6849-4fdd-b08a-ba7789963e94" (UID: "621261fa-6849-4fdd-b08a-ba7789963e94"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 20:28:23.302227 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.302195 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-istio-envoy\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:28:23.302227 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.302223 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbljg\" (UniqueName: \"kubernetes.io/projected/621261fa-6849-4fdd-b08a-ba7789963e94-kube-api-access-vbljg\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:28:23.302227 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.302232 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/621261fa-6849-4fdd-b08a-ba7789963e94-istiod-ca-cert\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:28:23.302472 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.302242 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/621261fa-6849-4fdd-b08a-ba7789963e94-istio-data\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:28:23.302472 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.302250 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/621261fa-6849-4fdd-b08a-ba7789963e94-istio-podinfo\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:28:23.302472 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.302257 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/621261fa-6849-4fdd-b08a-ba7789963e94-istio-token\") on node \"ip-10-0-130-172.ec2.internal\" DevicePath \"\"" Jul 06 20:28:23.324725 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.324648 2581 generic.go:356] "Generic (PLEG): container finished" podID="621261fa-6849-4fdd-b08a-ba7789963e94" containerID="6134980c09a4f5426e3c9a487240aa3e76307212c1ab0cefacda46f59cff1042" exitCode=0 Jul 06 20:28:23.324725 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.324699 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" Jul 06 20:28:23.324884 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.324730 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" event={"ID":"621261fa-6849-4fdd-b08a-ba7789963e94","Type":"ContainerDied","Data":"6134980c09a4f5426e3c9a487240aa3e76307212c1ab0cefacda46f59cff1042"} Jul 06 20:28:23.324884 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.324786 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r" event={"ID":"621261fa-6849-4fdd-b08a-ba7789963e94","Type":"ContainerDied","Data":"c2a0c99fd1bb0f7dc557f7262edacf6f990542ff041d17bc4d3e767a203a793c"} Jul 06 20:28:23.324884 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.324817 2581 scope.go:117] "RemoveContainer" containerID="6134980c09a4f5426e3c9a487240aa3e76307212c1ab0cefacda46f59cff1042" Jul 06 20:28:23.335204 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.335177 2581 scope.go:117] "RemoveContainer" containerID="14c7501dcbc56235271e6ff1ed9f7cfccfa53cefa06580a24ce19d1673a0de82" Jul 06 20:28:23.346058 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.346039 2581 scope.go:117] "RemoveContainer" containerID="6134980c09a4f5426e3c9a487240aa3e76307212c1ab0cefacda46f59cff1042" Jul 06 20:28:23.346359 ip-10-0-130-172 kubenswrapper[2581]: E0706 20:28:23.346336 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6134980c09a4f5426e3c9a487240aa3e76307212c1ab0cefacda46f59cff1042\": container with ID starting with 6134980c09a4f5426e3c9a487240aa3e76307212c1ab0cefacda46f59cff1042 not found: ID does not exist" containerID="6134980c09a4f5426e3c9a487240aa3e76307212c1ab0cefacda46f59cff1042" Jul 06 20:28:23.346471 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.346370 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6134980c09a4f5426e3c9a487240aa3e76307212c1ab0cefacda46f59cff1042"} err="failed to get container status \"6134980c09a4f5426e3c9a487240aa3e76307212c1ab0cefacda46f59cff1042\": rpc error: code = NotFound desc = could not find container \"6134980c09a4f5426e3c9a487240aa3e76307212c1ab0cefacda46f59cff1042\": container with ID starting with 6134980c09a4f5426e3c9a487240aa3e76307212c1ab0cefacda46f59cff1042 not found: ID does not exist" Jul 06 20:28:23.346471 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.346393 2581 scope.go:117] "RemoveContainer" containerID="14c7501dcbc56235271e6ff1ed9f7cfccfa53cefa06580a24ce19d1673a0de82" Jul 06 20:28:23.346643 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.346624 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r"] Jul 06 20:28:23.346681 ip-10-0-130-172 kubenswrapper[2581]: E0706 20:28:23.346645 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14c7501dcbc56235271e6ff1ed9f7cfccfa53cefa06580a24ce19d1673a0de82\": container with ID starting with 14c7501dcbc56235271e6ff1ed9f7cfccfa53cefa06580a24ce19d1673a0de82 not found: ID does not exist" containerID="14c7501dcbc56235271e6ff1ed9f7cfccfa53cefa06580a24ce19d1673a0de82" Jul 06 20:28:23.346681 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.346665 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14c7501dcbc56235271e6ff1ed9f7cfccfa53cefa06580a24ce19d1673a0de82"} err="failed to get container status \"14c7501dcbc56235271e6ff1ed9f7cfccfa53cefa06580a24ce19d1673a0de82\": rpc error: code = NotFound desc = could not find container \"14c7501dcbc56235271e6ff1ed9f7cfccfa53cefa06580a24ce19d1673a0de82\": container with ID starting with 14c7501dcbc56235271e6ff1ed9f7cfccfa53cefa06580a24ce19d1673a0de82 not found: ID does not exist" Jul 06 20:28:23.351108 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.351086 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-a2776d-openshift-default-79748b66cf-mb24r"] Jul 06 20:28:23.501771 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.501739 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-f2849_442fc246-0e4b-40cb-9cb9-4d11286a5676/manager/0.log" Jul 06 20:28:23.634929 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.634854 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-t9ntm_ff9f8daa-34bd-47f2-84ab-818793008a03/manager/0.log" Jul 06 20:28:23.756427 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.756395 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5846b87474-2rgtl_28982306-24aa-4afa-926e-05866cc42c51/maas-api/0.log" Jul 06 20:28:23.872519 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:23.872492 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722800-f5mtb_12c5da94-35d7-4262-8cf1-6cb83ba4b60f/cleanup/0.log" Jul 06 20:28:24.110827 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:24.110797 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7588cc9ccd-fm8tf_0111e398-087a-4d2a-8a38-e08a03c406f6/manager/0.log" Jul 06 20:28:24.490791 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:24.490725 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74dff95f86-gnhkm_f412df03-f5f8-4647-bd8e-beb4c9e6a60f/copy-manifests/0.log" Jul 06 20:28:24.498450 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:24.498433 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74dff95f86-gnhkm_f412df03-f5f8-4647-bd8e-beb4c9e6a60f/manager/0.log" Jul 06 20:28:24.866574 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:24.866548 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-vzglc_91550056-791c-447d-8bf2-c83d2b11e539/postgres/0.log" Jul 06 20:28:25.095577 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:25.095547 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="621261fa-6849-4fdd-b08a-ba7789963e94" path="/var/lib/kubelet/pods/621261fa-6849-4fdd-b08a-ba7789963e94/volumes" Jul 06 20:28:26.983454 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:26.983423 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-phmfd_c08eece9-6a10-4b09-a4c3-54389d993c67/manager/0.log" Jul 06 20:28:27.590383 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:27.590351 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4hlkx6_2472da42-8346-454f-ae81-7cdea6bbb2c5/istio-proxy/0.log" Jul 06 20:28:28.058077 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:28.058050 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-lc6bh_266f87bd-dd56-4772-8d0d-4cfe66e673fb/istio-proxy/1.log" Jul 06 20:28:28.198231 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:28.198190 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-kv56k_9137adb9-e557-4709-9f69-ab31edf80048/payload-pre-processing/0.log" Jul 06 20:28:28.328624 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:28.328550 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-44wkr_a5f3b493-9b0f-44d8-aba5-257b0c46f540/payload-processing/0.log" Jul 06 20:28:28.815991 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:28.815963 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-qczfq_c6db7aaf-1c96-48ad-94b5-472c2b6b36b1/main/0.log" Jul 06 20:28:29.173535 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:29.173466 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbpbcfd_4095e67f-7cd4-45bf-8b4e-860ae54e8f9c/main/0.log" Jul 06 20:28:36.612608 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:36.612578 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9qrkq_baaf2afb-f6b8-41e2-8ef5-a0e1e8ed03e6/global-pull-secret-syncer/0.log" Jul 06 20:28:36.814254 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:36.814220 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zwmlp_a59fb421-1d29-4686-8512-af1ee3473b27/konnectivity-agent/0.log" Jul 06 20:28:36.836101 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:36.836075 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-172.ec2.internal_8812863089e77e28db706ad567f2f4d4/haproxy/0.log" Jul 06 20:28:42.159897 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:42.159859 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-phmfd_c08eece9-6a10-4b09-a4c3-54389d993c67/manager/0.log" Jul 06 20:28:43.888507 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:43.888474 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1687c4c2-7af9-479e-a31f-dbc13a04c41c/alertmanager/0.log" Jul 06 20:28:43.912874 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:43.912805 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1687c4c2-7af9-479e-a31f-dbc13a04c41c/config-reloader/0.log" Jul 06 20:28:43.940155 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:43.940129 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1687c4c2-7af9-479e-a31f-dbc13a04c41c/kube-rbac-proxy-web/0.log" Jul 06 20:28:43.965294 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:43.965274 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1687c4c2-7af9-479e-a31f-dbc13a04c41c/kube-rbac-proxy/0.log" Jul 06 20:28:43.990152 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:43.990123 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1687c4c2-7af9-479e-a31f-dbc13a04c41c/kube-rbac-proxy-metric/0.log" Jul 06 20:28:44.012590 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.012565 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1687c4c2-7af9-479e-a31f-dbc13a04c41c/prom-label-proxy/0.log" Jul 06 20:28:44.034283 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.034256 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1687c4c2-7af9-479e-a31f-dbc13a04c41c/init-config-reloader/0.log" Jul 06 20:28:44.101793 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.101763 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-cpt55_c6dc6c9f-da74-41d5-80c4-08baf2959a41/kube-state-metrics/0.log" Jul 06 20:28:44.126201 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.126174 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-cpt55_c6dc6c9f-da74-41d5-80c4-08baf2959a41/kube-rbac-proxy-main/0.log" Jul 06 20:28:44.160803 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.160776 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-cpt55_c6dc6c9f-da74-41d5-80c4-08baf2959a41/kube-rbac-proxy-self/0.log" Jul 06 20:28:44.278506 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.278451 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-nfrhc_85fe5aa1-7407-4735-aede-ef94e7bb4800/monitoring-plugin/0.log" Jul 06 20:28:44.473527 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.473491 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lssgd_b66e29b3-d6fd-4dd6-9110-5277755d2bba/node-exporter/0.log" Jul 06 20:28:44.497477 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.497452 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lssgd_b66e29b3-d6fd-4dd6-9110-5277755d2bba/kube-rbac-proxy/0.log" Jul 06 20:28:44.519791 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.519765 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lssgd_b66e29b3-d6fd-4dd6-9110-5277755d2bba/init-textfile/0.log" Jul 06 20:28:44.545490 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.545401 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-8wf5j_100b14a8-13b5-48bc-a303-896ff7743bed/kube-rbac-proxy-main/0.log" Jul 06 20:28:44.570296 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.570274 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-8wf5j_100b14a8-13b5-48bc-a303-896ff7743bed/kube-rbac-proxy-self/0.log" Jul 06 20:28:44.594801 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.594768 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-8wf5j_100b14a8-13b5-48bc-a303-896ff7743bed/openshift-state-metrics/0.log" Jul 06 20:28:44.642980 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.642951 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e38060bc-ae18-423c-9466-b62298ae81c4/prometheus/0.log" Jul 06 20:28:44.666537 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.666515 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e38060bc-ae18-423c-9466-b62298ae81c4/config-reloader/0.log" Jul 06 20:28:44.690698 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.690671 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e38060bc-ae18-423c-9466-b62298ae81c4/thanos-sidecar/0.log" Jul 06 20:28:44.714417 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.714390 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e38060bc-ae18-423c-9466-b62298ae81c4/kube-rbac-proxy-web/0.log" Jul 06 20:28:44.741841 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.741808 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e38060bc-ae18-423c-9466-b62298ae81c4/kube-rbac-proxy/0.log" Jul 06 20:28:44.765494 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.765467 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e38060bc-ae18-423c-9466-b62298ae81c4/kube-rbac-proxy-thanos/0.log" Jul 06 20:28:44.788746 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.788722 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e38060bc-ae18-423c-9466-b62298ae81c4/init-config-reloader/0.log" Jul 06 20:28:44.867451 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.867358 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-794xr_ab738938-c8aa-4c7d-b230-6e0b385079d6/prometheus-operator-admission-webhook/0.log" Jul 06 20:28:44.897904 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.897871 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54f8cd97b5-sjsdh_22715c7b-7254-4d6c-a760-2c58e4f071a7/telemeter-client/0.log" Jul 06 20:28:44.920164 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.920138 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54f8cd97b5-sjsdh_22715c7b-7254-4d6c-a760-2c58e4f071a7/reload/0.log" Jul 06 20:28:44.944563 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:44.944542 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54f8cd97b5-sjsdh_22715c7b-7254-4d6c-a760-2c58e4f071a7/kube-rbac-proxy/0.log" Jul 06 20:28:46.308030 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.307951 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj"] Jul 06 20:28:46.308467 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.308420 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a282e56-d009-4cac-85ad-7536563a486e" containerName="istio-proxy" Jul 06 20:28:46.308467 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.308439 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a282e56-d009-4cac-85ad-7536563a486e" containerName="istio-proxy" Jul 06 20:28:46.308467 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.308450 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" containerName="istio-proxy" Jul 06 20:28:46.308467 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.308458 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" containerName="istio-proxy" Jul 06 20:28:46.308672 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.308492 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="621261fa-6849-4fdd-b08a-ba7789963e94" containerName="istio-proxy" Jul 06 20:28:46.308672 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.308500 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="621261fa-6849-4fdd-b08a-ba7789963e94" containerName="istio-proxy" Jul 06 20:28:46.308672 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.308526 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="621261fa-6849-4fdd-b08a-ba7789963e94" containerName="istio-proxy" Jul 06 20:28:46.308672 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.308534 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="621261fa-6849-4fdd-b08a-ba7789963e94" containerName="istio-proxy" Jul 06 20:28:46.308672 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.308630 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" containerName="istio-proxy" Jul 06 20:28:46.308672 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.308643 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="621261fa-6849-4fdd-b08a-ba7789963e94" containerName="istio-proxy" Jul 06 20:28:46.308672 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.308650 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6a9c990-9d8c-4879-8ce5-20cfce81f7e9" containerName="istio-proxy" Jul 06 20:28:46.312245 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.312229 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj" Jul 06 20:28:46.315013 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.314970 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4b4qp\"/\"default-dockercfg-djtng\"" Jul 06 20:28:46.315986 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.315961 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4b4qp\"/\"kube-root-ca.crt\"" Jul 06 20:28:46.316110 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.315989 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4b4qp\"/\"openshift-service-ca.crt\"" Jul 06 20:28:46.330943 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.330918 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj"] Jul 06 20:28:46.429453 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.429419 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/16854f58-3718-4538-b0d0-3bf6e34cf25e-proc\") pod \"perf-node-gather-daemonset-w94sj\" (UID: \"16854f58-3718-4538-b0d0-3bf6e34cf25e\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj" Jul 06 20:28:46.429641 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.429523 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/16854f58-3718-4538-b0d0-3bf6e34cf25e-podres\") pod \"perf-node-gather-daemonset-w94sj\" (UID: \"16854f58-3718-4538-b0d0-3bf6e34cf25e\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj" Jul 06 20:28:46.429641 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.429627 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16854f58-3718-4538-b0d0-3bf6e34cf25e-lib-modules\") pod \"perf-node-gather-daemonset-w94sj\" (UID: \"16854f58-3718-4538-b0d0-3bf6e34cf25e\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj" Jul 06 20:28:46.429717 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.429650 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js225\" (UniqueName: \"kubernetes.io/projected/16854f58-3718-4538-b0d0-3bf6e34cf25e-kube-api-access-js225\") pod \"perf-node-gather-daemonset-w94sj\" (UID: \"16854f58-3718-4538-b0d0-3bf6e34cf25e\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj" Jul 06 20:28:46.429717 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.429665 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16854f58-3718-4538-b0d0-3bf6e34cf25e-sys\") pod \"perf-node-gather-daemonset-w94sj\" (UID: \"16854f58-3718-4538-b0d0-3bf6e34cf25e\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj" Jul 06 20:28:46.530459 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.530417 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16854f58-3718-4538-b0d0-3bf6e34cf25e-lib-modules\") pod \"perf-node-gather-daemonset-w94sj\" (UID: \"16854f58-3718-4538-b0d0-3bf6e34cf25e\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj" Jul 06 20:28:46.530653 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.530480 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js225\" (UniqueName: \"kubernetes.io/projected/16854f58-3718-4538-b0d0-3bf6e34cf25e-kube-api-access-js225\") pod \"perf-node-gather-daemonset-w94sj\" (UID: \"16854f58-3718-4538-b0d0-3bf6e34cf25e\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj" Jul 06 20:28:46.530653 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.530500 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16854f58-3718-4538-b0d0-3bf6e34cf25e-sys\") pod \"perf-node-gather-daemonset-w94sj\" (UID: \"16854f58-3718-4538-b0d0-3bf6e34cf25e\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj" Jul 06 20:28:46.530653 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.530523 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/16854f58-3718-4538-b0d0-3bf6e34cf25e-proc\") pod \"perf-node-gather-daemonset-w94sj\" (UID: \"16854f58-3718-4538-b0d0-3bf6e34cf25e\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj" Jul 06 20:28:46.530653 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.530598 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/16854f58-3718-4538-b0d0-3bf6e34cf25e-podres\") pod \"perf-node-gather-daemonset-w94sj\" (UID: \"16854f58-3718-4538-b0d0-3bf6e34cf25e\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj" Jul 06 20:28:46.530653 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.530603 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/16854f58-3718-4538-b0d0-3bf6e34cf25e-lib-modules\") pod \"perf-node-gather-daemonset-w94sj\" (UID: \"16854f58-3718-4538-b0d0-3bf6e34cf25e\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj" Jul 06 20:28:46.530653 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.530618 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16854f58-3718-4538-b0d0-3bf6e34cf25e-sys\") pod \"perf-node-gather-daemonset-w94sj\" (UID: \"16854f58-3718-4538-b0d0-3bf6e34cf25e\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj" Jul 06 20:28:46.530920 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.530703 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/16854f58-3718-4538-b0d0-3bf6e34cf25e-proc\") pod \"perf-node-gather-daemonset-w94sj\" (UID: \"16854f58-3718-4538-b0d0-3bf6e34cf25e\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj" Jul 06 20:28:46.530920 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.530725 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/16854f58-3718-4538-b0d0-3bf6e34cf25e-podres\") pod \"perf-node-gather-daemonset-w94sj\" (UID: \"16854f58-3718-4538-b0d0-3bf6e34cf25e\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj" Jul 06 20:28:46.537945 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.537904 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-js225\" (UniqueName: \"kubernetes.io/projected/16854f58-3718-4538-b0d0-3bf6e34cf25e-kube-api-access-js225\") pod \"perf-node-gather-daemonset-w94sj\" (UID: \"16854f58-3718-4538-b0d0-3bf6e34cf25e\") " pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj" Jul 06 20:28:46.622890 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.622797 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj" Jul 06 20:28:46.767281 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.767253 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj"] Jul 06 20:28:46.768796 ip-10-0-130-172 kubenswrapper[2581]: W0706 20:28:46.768766 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod16854f58_3718_4538_b0d0_3bf6e34cf25e.slice/crio-f6ddd73c619527a3a5152debab5292ce4d580e2e5496e0367969189339e76b52 WatchSource:0}: Error finding container f6ddd73c619527a3a5152debab5292ce4d580e2e5496e0367969189339e76b52: Status 404 returned error can't find the container with id f6ddd73c619527a3a5152debab5292ce4d580e2e5496e0367969189339e76b52 Jul 06 20:28:46.806917 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.806889 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/2.log" Jul 06 20:28:46.811749 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:46.811724 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-2xj4r_25615b1c-46b5-484c-a198-88e9aac6fe21/console-operator/3.log" Jul 06 20:28:47.454593 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:47.454554 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj" event={"ID":"16854f58-3718-4538-b0d0-3bf6e34cf25e","Type":"ContainerStarted","Data":"82186030faf691a9bf1d587119945eb353c40a6f1e768556974eddf305ebf863"} Jul 06 20:28:47.455091 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:47.454614 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj" event={"ID":"16854f58-3718-4538-b0d0-3bf6e34cf25e","Type":"ContainerStarted","Data":"f6ddd73c619527a3a5152debab5292ce4d580e2e5496e0367969189339e76b52"} Jul 06 20:28:47.455091 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:47.454674 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj" Jul 06 20:28:47.484534 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:47.484481 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj" podStartSLOduration=1.484463253 podStartE2EDuration="1.484463253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:28:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:28:47.482898661 +0000 UTC m=+2426.930135383" watchObservedRunningTime="2026-07-06 20:28:47.484463253 +0000 UTC m=+2426.931699969" Jul 06 20:28:48.923275 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:48.923243 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r9czf_19b3e6d4-9980-4130-bff7-3a9f41085938/dns/0.log" Jul 06 20:28:48.945272 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:48.945253 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r9czf_19b3e6d4-9980-4130-bff7-3a9f41085938/kube-rbac-proxy/0.log" Jul 06 20:28:48.995941 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:48.995919 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gr86q_ad86aa92-d126-4e69-a979-247c6deb986f/dns-node-resolver/0.log" Jul 06 20:28:49.581277 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:49.581243 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ldkxm_03659b3b-23b9-4ce3-9c69-d76c580fd1b7/node-ca/0.log" Jul 06 20:28:50.498923 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:50.498893 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4hlkx6_2472da42-8346-454f-ae81-7cdea6bbb2c5/istio-proxy/0.log" Jul 06 20:28:50.685102 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:50.685070 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-lc6bh_266f87bd-dd56-4772-8d0d-4cfe66e673fb/istio-proxy/0.log" Jul 06 20:28:50.711631 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:50.711575 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-lc6bh_266f87bd-dd56-4772-8d0d-4cfe66e673fb/istio-proxy/1.log" Jul 06 20:28:50.742410 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:50.742376 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-kv56k_9137adb9-e557-4709-9f69-ab31edf80048/payload-pre-processing/0.log" Jul 06 20:28:50.788180 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:50.788146 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-44wkr_a5f3b493-9b0f-44d8-aba5-257b0c46f540/payload-processing/0.log" Jul 06 20:28:51.474889 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:51.474857 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8jbg4_8ba1d496-a1a9-4f60-a589-9ae3946d39b9/serve-healthcheck-canary/0.log" Jul 06 20:28:52.078177 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:52.078148 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lkt7q_4ac0796b-7777-4b33-8a0e-a369697d8d88/kube-rbac-proxy/0.log" Jul 06 20:28:52.100511 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:52.100487 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lkt7q_4ac0796b-7777-4b33-8a0e-a369697d8d88/exporter/0.log" Jul 06 20:28:52.122766 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:52.122735 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lkt7q_4ac0796b-7777-4b33-8a0e-a369697d8d88/extractor/0.log" Jul 06 20:28:53.468770 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:53.468742 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4b4qp/perf-node-gather-daemonset-w94sj" Jul 06 20:28:54.257268 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:54.257237 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-f2849_442fc246-0e4b-40cb-9cb9-4d11286a5676/manager/0.log" Jul 06 20:28:54.382022 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:54.381989 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-t9ntm_ff9f8daa-34bd-47f2-84ab-818793008a03/manager/0.log" Jul 06 20:28:54.460762 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:54.460729 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5846b87474-2rgtl_28982306-24aa-4afa-926e-05866cc42c51/maas-api/0.log" Jul 06 20:28:54.503091 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:54.503064 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722800-f5mtb_12c5da94-35d7-4262-8cf1-6cb83ba4b60f/cleanup/0.log" Jul 06 20:28:54.633355 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:54.633256 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7588cc9ccd-fm8tf_0111e398-087a-4d2a-8a38-e08a03c406f6/manager/0.log" Jul 06 20:28:54.788626 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:54.788597 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74dff95f86-gnhkm_f412df03-f5f8-4647-bd8e-beb4c9e6a60f/manager/0.log" Jul 06 20:28:54.815675 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:54.815657 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-74dff95f86-gnhkm_f412df03-f5f8-4647-bd8e-beb4c9e6a60f/copy-manifests/0.log" Jul 06 20:28:54.987951 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:54.987869 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-vzglc_91550056-791c-447d-8bf2-c83d2b11e539/postgres/0.log" Jul 06 20:28:56.478417 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:56.478385 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-59cb764c9c-qvj5k_2f04ef4f-fea0-4bf7-871d-5162dfcd6cd1/manager/0.log" Jul 06 20:28:56.503260 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:56.503237 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-zcv8d_d988356c-af68-4828-be93-027302fdabb4/openshift-lws-operator/0.log" Jul 06 20:28:56.522329 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:56.522288 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4hlkx6_2472da42-8346-454f-ae81-7cdea6bbb2c5/istio-proxy/0.log" Jul 06 20:28:56.665108 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:56.665037 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-lc6bh_266f87bd-dd56-4772-8d0d-4cfe66e673fb/istio-proxy/0.log" Jul 06 20:28:56.692341 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:56.692297 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-lc6bh_266f87bd-dd56-4772-8d0d-4cfe66e673fb/istio-proxy/1.log" Jul 06 20:28:56.713192 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:56.713168 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-kv56k_9137adb9-e557-4709-9f69-ab31edf80048/payload-pre-processing/0.log" Jul 06 20:28:56.754682 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:56.754652 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-44wkr_a5f3b493-9b0f-44d8-aba5-257b0c46f540/payload-processing/0.log" Jul 06 20:28:57.958592 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:57.958566 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4hlkx6_2472da42-8346-454f-ae81-7cdea6bbb2c5/istio-proxy/0.log" Jul 06 20:28:58.109676 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:58.109642 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-lc6bh_266f87bd-dd56-4772-8d0d-4cfe66e673fb/istio-proxy/0.log" Jul 06 20:28:58.138000 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:58.137966 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-lc6bh_266f87bd-dd56-4772-8d0d-4cfe66e673fb/istio-proxy/1.log" Jul 06 20:28:58.161161 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:58.161088 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-kv56k_9137adb9-e557-4709-9f69-ab31edf80048/payload-pre-processing/0.log" Jul 06 20:28:58.200436 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:28:58.200408 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-44wkr_a5f3b493-9b0f-44d8-aba5-257b0c46f540/payload-processing/0.log" Jul 06 20:29:01.153183 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:29:01.153154 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-9jm7x_da05b35d-a058-47bc-bf23-e87c5fbbe9b9/migrator/0.log" Jul 06 20:29:01.173744 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:29:01.173719 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-9jm7x_da05b35d-a058-47bc-bf23-e87c5fbbe9b9/graceful-termination/0.log" Jul 06 20:29:02.645072 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:29:02.644996 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wlcd6_88cb2c01-723f-4d6f-a264-8f6a110a1f6e/kube-multus-additional-cni-plugins/0.log" Jul 06 20:29:02.670400 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:29:02.670376 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wlcd6_88cb2c01-723f-4d6f-a264-8f6a110a1f6e/egress-router-binary-copy/0.log" Jul 06 20:29:02.691897 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:29:02.691869 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wlcd6_88cb2c01-723f-4d6f-a264-8f6a110a1f6e/cni-plugins/0.log" Jul 06 20:29:02.713128 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:29:02.713101 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wlcd6_88cb2c01-723f-4d6f-a264-8f6a110a1f6e/bond-cni-plugin/0.log" Jul 06 20:29:02.737788 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:29:02.737761 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wlcd6_88cb2c01-723f-4d6f-a264-8f6a110a1f6e/routeoverride-cni/0.log" Jul 06 20:29:02.759346 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:29:02.759307 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wlcd6_88cb2c01-723f-4d6f-a264-8f6a110a1f6e/whereabouts-cni-bincopy/0.log" Jul 06 20:29:02.780344 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:29:02.780288 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wlcd6_88cb2c01-723f-4d6f-a264-8f6a110a1f6e/whereabouts-cni/0.log" Jul 06 20:29:02.983461 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:29:02.983383 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-cktbc_ca05caa7-6d81-4a18-bc8e-d1070594cb2c/kube-multus/0.log" Jul 06 20:29:03.162012 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:29:03.161976 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tgl28_2df21624-1cb1-4437-99a7-b91b4784d32d/network-metrics-daemon/0.log" Jul 06 20:29:03.181006 ip-10-0-130-172 kubenswrapper[2581]: I0706 20:29:03.180978 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tgl28_2df21624-1cb1-4437-99a7-b91b4784d32d/kube-rbac-proxy/0.log"