Jul 07 14:58:42.219600 ip-10-0-130-39 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 07 14:58:42.219611 ip-10-0-130-39 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 07 14:58:42.219618 ip-10-0-130-39 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 07 14:58:42.219925 ip-10-0-130-39 systemd[1]: Failed to start Kubernetes Kubelet. Jul 07 14:58:52.468487 ip-10-0-130-39 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 07 14:58:52.468506 ip-10-0-130-39 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 8ca86b7c4f69433293bd1714d0648f0a -- Jul 07 15:00:51.064167 ip-10-0-130-39 systemd[1]: Starting Kubernetes Kubelet... Jul 07 15:00:51.521652 ip-10-0-130-39 kubenswrapper[2556]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 15:00:51.521652 ip-10-0-130-39 kubenswrapper[2556]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 15:00:51.521652 ip-10-0-130-39 kubenswrapper[2556]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 15:00:51.521652 ip-10-0-130-39 kubenswrapper[2556]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 15:00:51.521652 ip-10-0-130-39 kubenswrapper[2556]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 15:00:51.521652 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.475223 2556 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 15:00:51.521652 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481653 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:00:51.521652 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481668 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:00:51.521652 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481671 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:00:51.521652 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481675 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:00:51.521652 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481678 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:00:51.521652 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481681 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:00:51.521652 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481683 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:00:51.521652 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481686 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:00:51.522478 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481688 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:00:51.522478 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481691 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:00:51.522478 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481694 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:00:51.522478 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481697 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:00:51.522478 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481701 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:00:51.522478 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481703 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:00:51.522478 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481706 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:00:51.522478 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481708 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:00:51.522478 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481711 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:00:51.522478 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481714 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:00:51.522478 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481717 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:00:51.522478 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481719 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:00:51.522478 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481722 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:00:51.522478 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481724 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:00:51.522478 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481727 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:00:51.522478 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481730 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:00:51.522478 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481732 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:00:51.522478 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481734 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:00:51.522478 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481737 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:00:51.522478 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481740 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:00:51.524666 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481749 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:00:51.524666 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481752 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:00:51.524666 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481755 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:00:51.524666 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481763 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:00:51.524666 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481765 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:00:51.524666 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481768 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:00:51.524666 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481771 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:00:51.524666 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481773 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:00:51.524666 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481777 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:00:51.524666 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481782 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:00:51.524666 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481784 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:00:51.524666 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481787 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:00:51.524666 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481789 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:00:51.524666 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481792 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:00:51.524666 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481795 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:00:51.524666 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481799 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:00:51.524666 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481801 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:00:51.524666 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481805 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:00:51.524666 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481810 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:00:51.525485 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481814 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:00:51.525485 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481817 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:00:51.525485 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481820 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:00:51.525485 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481823 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:00:51.525485 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481826 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:00:51.525485 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481829 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:00:51.525485 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481832 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:00:51.525485 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481834 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:00:51.525485 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481837 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:00:51.525485 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481839 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:00:51.525485 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481842 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:00:51.525485 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481845 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:00:51.525485 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481848 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:00:51.525485 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481850 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:00:51.525485 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481855 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:00:51.525485 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481858 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:00:51.525485 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481861 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:00:51.525485 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481879 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:00:51.525485 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481884 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:00:51.526756 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481889 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:00:51.526756 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481892 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:00:51.526756 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481895 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:00:51.526756 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481898 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:00:51.526756 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481901 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:00:51.526756 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481903 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:00:51.526756 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481906 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:00:51.526756 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481908 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:00:51.526756 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481911 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:00:51.526756 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481913 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:00:51.526756 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481916 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:00:51.526756 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481918 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:00:51.526756 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481921 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:00:51.526756 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481923 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:00:51.526756 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481926 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:00:51.526756 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481928 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:00:51.526756 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481931 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:00:51.526756 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481935 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:00:51.526756 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481938 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:00:51.527585 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481940 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:00:51.527585 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481943 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:00:51.527585 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481946 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:00:51.527585 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481949 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:00:51.527585 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481951 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:00:51.527585 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481954 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:00:51.527585 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481957 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:00:51.527585 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481959 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:00:51.527585 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481962 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:00:51.527585 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481964 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:00:51.527585 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481967 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:00:51.527585 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481969 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:00:51.527585 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481972 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:00:51.527585 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.481975 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:00:51.527585 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482684 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:00:51.527585 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482690 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:00:51.527585 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482694 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:00:51.527585 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482697 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:00:51.527585 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482700 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:00:51.528809 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482702 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:00:51.528809 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482705 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:00:51.528809 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482707 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:00:51.528809 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482710 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:00:51.528809 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482713 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:00:51.528809 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482716 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:00:51.528809 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482718 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:00:51.528809 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482721 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:00:51.528809 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482723 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:00:51.528809 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482727 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:00:51.528809 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482729 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:00:51.528809 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482732 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:00:51.528809 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482736 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:00:51.528809 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482739 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:00:51.528809 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482741 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:00:51.528809 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482744 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:00:51.528809 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482746 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:00:51.528809 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482749 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:00:51.528809 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482752 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:00:51.528809 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482755 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:00:51.531325 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482757 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:00:51.531325 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482760 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:00:51.531325 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482762 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:00:51.531325 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482765 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:00:51.531325 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482767 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:00:51.531325 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482770 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:00:51.531325 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482773 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:00:51.531325 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482775 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:00:51.531325 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482778 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:00:51.531325 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482780 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:00:51.531325 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482785 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:00:51.531325 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482789 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:00:51.531325 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482792 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:00:51.531325 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482794 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:00:51.531325 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482797 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:00:51.531325 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482800 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:00:51.531325 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482802 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:00:51.531325 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482804 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:00:51.531325 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482807 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:00:51.531325 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482810 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:00:51.531923 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482813 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:00:51.531923 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482815 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:00:51.531923 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482818 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:00:51.531923 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482821 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:00:51.531923 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482823 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:00:51.531923 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482826 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:00:51.531923 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482829 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:00:51.531923 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482831 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:00:51.531923 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482835 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:00:51.531923 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482838 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:00:51.531923 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482840 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:00:51.531923 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482843 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:00:51.531923 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482845 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:00:51.531923 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482847 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:00:51.531923 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482850 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:00:51.531923 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482852 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:00:51.531923 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482855 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:00:51.531923 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482858 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:00:51.531923 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482860 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:00:51.532484 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482862 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:00:51.532484 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482880 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:00:51.532484 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482883 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:00:51.532484 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482886 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:00:51.532484 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482889 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:00:51.532484 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482892 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:00:51.532484 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482894 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:00:51.532484 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482897 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:00:51.532484 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482899 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:00:51.532484 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482902 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:00:51.532484 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482904 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:00:51.532484 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482907 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:00:51.532484 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482910 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:00:51.532484 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482916 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:00:51.532484 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482919 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:00:51.532484 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482922 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:00:51.532484 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482925 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:00:51.532484 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482928 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:00:51.532484 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482931 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:00:51.532484 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482934 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:00:51.533146 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482937 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:00:51.533146 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482939 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:00:51.533146 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482942 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:00:51.533146 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482945 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:00:51.533146 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482947 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:00:51.533146 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482950 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:00:51.533146 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482952 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:00:51.533146 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482955 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:00:51.533146 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482957 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:00:51.533146 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482959 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:00:51.533146 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482964 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:00:51.533146 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482967 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:00:51.533146 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482970 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:00:51.533146 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482973 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:00:51.533146 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.482975 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:00:51.533146 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483372 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:00:51.533146 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483377 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:00:51.533146 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483379 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:00:51.533146 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483382 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:00:51.533716 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483386 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:00:51.533716 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483393 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:00:51.533716 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483396 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:00:51.533716 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483399 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:00:51.533716 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483402 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:00:51.533716 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483405 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:00:51.533716 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483408 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:00:51.533716 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483411 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:00:51.533716 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483414 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:00:51.533716 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483416 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:00:51.533716 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483419 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:00:51.533716 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483422 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:00:51.533716 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483425 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:00:51.533716 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483428 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:00:51.533716 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483431 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:00:51.533716 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483434 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:00:51.533716 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483436 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:00:51.533716 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483439 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:00:51.533716 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483441 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:00:51.533716 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483444 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:00:51.534286 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483446 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:00:51.534286 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483449 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:00:51.534286 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483452 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:00:51.534286 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483454 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:00:51.534286 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483456 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:00:51.534286 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483459 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:00:51.534286 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483461 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:00:51.534286 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483464 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:00:51.534286 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483467 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:00:51.534286 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483469 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:00:51.534286 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483472 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:00:51.534286 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483474 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:00:51.534286 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483477 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:00:51.534286 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483481 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:00:51.534286 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483484 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:00:51.534286 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483487 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:00:51.534286 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483489 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:00:51.534286 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483492 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:00:51.534286 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483495 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:00:51.534286 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483497 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:00:51.535099 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483500 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:00:51.535099 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483502 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:00:51.535099 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483505 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:00:51.535099 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483507 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:00:51.535099 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483510 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:00:51.535099 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483513 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:00:51.535099 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483521 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:00:51.535099 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483524 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:00:51.535099 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483526 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:00:51.535099 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483528 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:00:51.535099 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483532 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:00:51.535099 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483535 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:00:51.535099 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483538 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:00:51.535099 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483541 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:00:51.535099 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483543 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:00:51.535099 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483546 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:00:51.535099 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483549 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:00:51.535099 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483551 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:00:51.535099 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483554 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:00:51.535884 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483556 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:00:51.535884 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483559 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:00:51.535884 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483561 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:00:51.535884 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483564 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:00:51.535884 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483567 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:00:51.535884 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483569 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:00:51.535884 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483573 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:00:51.535884 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483576 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:00:51.535884 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483578 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:00:51.535884 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483581 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:00:51.535884 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483583 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:00:51.535884 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483586 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:00:51.535884 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483588 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:00:51.535884 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483593 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:00:51.535884 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483596 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:00:51.535884 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483599 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:00:51.535884 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483602 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:00:51.535884 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483605 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:00:51.535884 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483608 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:00:51.536669 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483613 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:00:51.536669 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483615 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:00:51.536669 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483618 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:00:51.536669 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483620 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:00:51.536669 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483622 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:00:51.536669 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483625 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:00:51.536669 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483627 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:00:51.536669 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483630 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:00:51.536669 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483632 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:00:51.536669 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483635 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:00:51.536669 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483637 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:00:51.536669 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483639 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:00:51.536669 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483642 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:00:51.536669 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483644 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:00:51.536669 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483647 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:00:51.536669 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483649 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:00:51.536669 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.483652 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:00:51.536669 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483737 2556 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 15:00:51.536669 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483745 2556 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 15:00:51.536669 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483753 2556 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 15:00:51.536669 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483757 2556 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483761 2556 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483764 2556 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483768 2556 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483772 2556 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483775 2556 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483778 2556 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483782 2556 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483785 2556 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483788 2556 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483791 2556 flags.go:64] FLAG: --cgroup-root="" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483794 2556 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483799 2556 flags.go:64] FLAG: --client-ca-file="" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483802 2556 flags.go:64] FLAG: --cloud-provider="external" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483805 2556 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483810 2556 flags.go:64] FLAG: --cluster-domain="" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483812 2556 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483816 2556 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483819 2556 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483822 2556 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483825 2556 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483828 2556 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483831 2556 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 15:00:51.537200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483835 2556 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483837 2556 flags.go:64] FLAG: --contention-profiling="false" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483840 2556 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483843 2556 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483847 2556 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483849 2556 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483853 2556 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483856 2556 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483860 2556 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483876 2556 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483880 2556 flags.go:64] FLAG: --enable-server="true" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483882 2556 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483887 2556 flags.go:64] FLAG: --event-burst="100" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483890 2556 flags.go:64] FLAG: --event-qps="50" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483892 2556 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483895 2556 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483898 2556 flags.go:64] FLAG: --eviction-hard="" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483902 2556 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483905 2556 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483907 2556 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483910 2556 flags.go:64] FLAG: --eviction-soft="" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483914 2556 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483917 2556 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483921 2556 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483924 2556 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 15:00:51.537885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483927 2556 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483930 2556 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483933 2556 flags.go:64] FLAG: --feature-gates="" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483936 2556 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483939 2556 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483942 2556 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483945 2556 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483948 2556 flags.go:64] FLAG: --healthz-port="10248" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483951 2556 flags.go:64] FLAG: --help="false" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483955 2556 flags.go:64] FLAG: --hostname-override="ip-10-0-130-39.ec2.internal" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483958 2556 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483961 2556 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483963 2556 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483967 2556 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483971 2556 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483975 2556 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483978 2556 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483981 2556 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483984 2556 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483987 2556 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483990 2556 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483993 2556 flags.go:64] FLAG: --kube-reserved="" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483995 2556 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.483998 2556 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 15:00:51.538622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484001 2556 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484004 2556 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484007 2556 flags.go:64] FLAG: --lock-file="" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484010 2556 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484014 2556 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484017 2556 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484022 2556 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484025 2556 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484028 2556 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484031 2556 flags.go:64] FLAG: --logging-format="text" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484034 2556 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484037 2556 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484040 2556 flags.go:64] FLAG: --manifest-url="" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484043 2556 flags.go:64] FLAG: --manifest-url-header="" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484047 2556 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484050 2556 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484055 2556 flags.go:64] FLAG: --max-pods="110" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484058 2556 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484061 2556 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484063 2556 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484066 2556 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484069 2556 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484072 2556 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484076 2556 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484084 2556 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 15:00:51.539259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484087 2556 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484090 2556 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484092 2556 flags.go:64] FLAG: --pod-cidr="" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484095 2556 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484100 2556 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484103 2556 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484106 2556 flags.go:64] FLAG: --pods-per-core="0" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484108 2556 flags.go:64] FLAG: --port="10250" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484111 2556 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484114 2556 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05ba4b0e2c379237e" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484117 2556 flags.go:64] FLAG: --qos-reserved="" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484121 2556 flags.go:64] FLAG: --read-only-port="10255" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484124 2556 flags.go:64] FLAG: --register-node="true" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484127 2556 flags.go:64] FLAG: --register-with-taints="" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484131 2556 flags.go:64] FLAG: --registry-burst="10" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484134 2556 flags.go:64] FLAG: --registry-qps="5" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484137 2556 flags.go:64] FLAG: --reserved-cpus="" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484140 2556 flags.go:64] FLAG: --reserved-memory="" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484143 2556 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484146 2556 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484149 2556 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484152 2556 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484155 2556 flags.go:64] FLAG: --runonce="false" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484157 2556 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484161 2556 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 15:00:51.540009 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484163 2556 flags.go:64] FLAG: --seccomp-default="false" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484166 2556 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484169 2556 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484172 2556 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484175 2556 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484178 2556 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484182 2556 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484185 2556 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484187 2556 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484190 2556 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484193 2556 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484196 2556 flags.go:64] FLAG: --system-cgroups="" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484199 2556 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484204 2556 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484207 2556 flags.go:64] FLAG: --tls-cert-file="" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484209 2556 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484213 2556 flags.go:64] FLAG: --tls-min-version="" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484216 2556 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484221 2556 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484223 2556 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484227 2556 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484230 2556 flags.go:64] FLAG: --v="2" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484234 2556 flags.go:64] FLAG: --version="false" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484238 2556 flags.go:64] FLAG: --vmodule="" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484242 2556 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 15:00:51.540612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.484245 2556 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 15:00:51.541415 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484357 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:00:51.541415 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484361 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:00:51.541415 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484364 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:00:51.541415 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484366 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:00:51.541415 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484369 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:00:51.541415 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484372 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:00:51.541415 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484374 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:00:51.541415 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484377 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:00:51.541415 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484379 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:00:51.541415 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484382 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:00:51.541415 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484384 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:00:51.541415 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484387 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:00:51.541415 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484392 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:00:51.541415 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484395 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:00:51.541415 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484398 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:00:51.541415 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484400 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:00:51.541415 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484403 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:00:51.541415 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484406 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:00:51.541415 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484408 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:00:51.541415 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484411 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:00:51.541924 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484413 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:00:51.541924 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484417 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:00:51.541924 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484420 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:00:51.541924 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484423 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:00:51.541924 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484427 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:00:51.541924 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484430 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:00:51.541924 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484433 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:00:51.541924 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484435 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:00:51.541924 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484438 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:00:51.541924 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484440 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:00:51.541924 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484444 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:00:51.541924 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484446 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:00:51.541924 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484449 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:00:51.541924 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484452 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:00:51.541924 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484454 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:00:51.541924 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484457 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:00:51.541924 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484459 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:00:51.541924 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484462 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:00:51.541924 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484464 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:00:51.542386 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484467 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:00:51.542386 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484470 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:00:51.542386 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484472 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:00:51.542386 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484475 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:00:51.542386 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484477 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:00:51.542386 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484481 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:00:51.542386 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484484 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:00:51.542386 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484486 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:00:51.542386 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484489 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:00:51.542386 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484492 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:00:51.542386 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484494 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:00:51.542386 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484496 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:00:51.542386 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484499 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:00:51.542386 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484501 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:00:51.542386 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484504 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:00:51.542386 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484507 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:00:51.542386 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484509 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:00:51.542386 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484513 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:00:51.542386 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484516 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:00:51.543089 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484518 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:00:51.543089 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484521 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:00:51.543089 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484523 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:00:51.543089 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484526 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:00:51.543089 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484528 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:00:51.543089 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484531 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:00:51.543089 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484533 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:00:51.543089 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484536 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:00:51.543089 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484540 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:00:51.543089 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484543 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:00:51.543089 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484546 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:00:51.543089 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484549 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:00:51.543089 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484552 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:00:51.543089 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484554 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:00:51.543089 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484557 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:00:51.543089 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484561 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:00:51.543089 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484564 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:00:51.543089 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484567 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:00:51.543089 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484570 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:00:51.543835 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484573 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:00:51.543835 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484575 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:00:51.543835 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484578 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:00:51.543835 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484581 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:00:51.543835 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484583 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:00:51.543835 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484586 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:00:51.543835 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484589 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:00:51.543835 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484591 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:00:51.543835 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484594 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:00:51.543835 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484596 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:00:51.543835 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484599 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:00:51.543835 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484603 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:00:51.543835 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484606 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:00:51.543835 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484608 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:00:51.543835 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484611 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:00:51.543835 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484613 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:00:51.543835 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484616 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:00:51.543835 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484619 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:00:51.543835 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484621 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:00:51.543835 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484624 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:00:51.543518 ip-10-0-130-39 systemd[1]: Started Kubernetes Kubelet. Jul 07 15:00:51.544594 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484627 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:00:51.544594 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.484629 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:00:51.544594 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.485252 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 15:00:51.544594 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.491839 2556 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 15:00:51.544594 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.491858 2556 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 15:00:51.544594 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.491891 2556 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 15:00:51.544594 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.491898 2556 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 15:00:51.544594 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.491954 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:00:51.544594 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.491958 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:00:51.544594 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.491962 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:00:51.544594 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.491965 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:00:51.544594 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.491967 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:00:51.544594 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.491972 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:00:51.544594 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.491975 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:00:51.544594 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.491978 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:00:51.545149 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.491981 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:00:51.545149 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.491983 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:00:51.545149 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.491986 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:00:51.545149 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.491989 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:00:51.545149 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.491992 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:00:51.545149 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.491994 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:00:51.545149 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.491997 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:00:51.545149 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492000 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:00:51.545149 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492004 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:00:51.545149 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492007 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:00:51.545149 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492010 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:00:51.545149 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492012 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:00:51.545149 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492015 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:00:51.545149 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492017 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:00:51.545149 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492020 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:00:51.545149 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492023 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:00:51.545149 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492025 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:00:51.545149 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492028 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:00:51.545149 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492031 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:00:51.545149 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492033 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:00:51.545772 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492036 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:00:51.545772 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492040 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:00:51.545772 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492043 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:00:51.545772 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492045 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:00:51.545772 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492048 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:00:51.545772 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492051 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:00:51.545772 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492053 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:00:51.545772 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492056 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:00:51.545772 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492059 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:00:51.545772 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492063 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:00:51.545772 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492065 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:00:51.545772 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492068 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:00:51.545772 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492070 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:00:51.545772 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492073 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:00:51.545772 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492076 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:00:51.545772 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492078 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:00:51.545772 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492081 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:00:51.545772 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492083 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:00:51.545772 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492086 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:00:51.545772 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492089 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:00:51.546391 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492093 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:00:51.546391 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492095 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:00:51.546391 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492098 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:00:51.546391 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492100 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:00:51.546391 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492103 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:00:51.546391 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492105 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:00:51.546391 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492108 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:00:51.546391 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492112 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:00:51.546391 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492116 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:00:51.546391 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492118 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:00:51.546391 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492121 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:00:51.546391 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492123 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:00:51.546391 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492126 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:00:51.546391 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492128 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:00:51.546391 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492132 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:00:51.546391 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492134 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:00:51.546391 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492137 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:00:51.546391 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492139 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:00:51.546391 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492142 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:00:51.547042 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492144 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:00:51.547042 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492147 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:00:51.547042 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492151 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:00:51.547042 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492153 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:00:51.547042 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492156 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:00:51.547042 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492158 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:00:51.547042 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492161 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:00:51.547042 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492163 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:00:51.547042 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492166 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:00:51.547042 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492168 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:00:51.547042 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492171 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:00:51.547042 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492173 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:00:51.547042 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492176 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:00:51.547042 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492180 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:00:51.547042 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492183 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:00:51.547042 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492185 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:00:51.547042 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492188 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:00:51.547042 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492191 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:00:51.547042 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492193 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:00:51.547042 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492195 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:00:51.547806 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492198 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:00:51.547806 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492201 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:00:51.547806 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492203 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:00:51.547806 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492207 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:00:51.547806 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492211 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:00:51.547806 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492214 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:00:51.547806 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492217 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:00:51.547806 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492220 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:00:51.547806 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492224 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:00:51.547806 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492228 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:00:51.547806 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492231 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:00:51.547806 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492234 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:00:51.547806 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.492240 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 15:00:51.547806 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492351 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 15:00:51.548388 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492355 2556 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 15:00:51.548388 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492360 2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 15:00:51.548388 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492363 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 15:00:51.548388 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492366 2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 15:00:51.548388 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492368 2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 15:00:51.548388 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492371 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 15:00:51.548388 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492373 2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 15:00:51.548388 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492376 2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 15:00:51.548388 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492379 2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 15:00:51.548388 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492382 2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 15:00:51.548388 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492385 2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 15:00:51.548388 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492389 2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 15:00:51.548388 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492391 2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 15:00:51.548388 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492394 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 15:00:51.548388 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492396 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 15:00:51.548388 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492399 2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 15:00:51.548388 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492402 2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 15:00:51.548388 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492404 2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 15:00:51.548388 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492407 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 15:00:51.548388 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492411 2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 15:00:51.549172 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492414 2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 15:00:51.549172 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492417 2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 15:00:51.549172 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492420 2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 15:00:51.549172 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492423 2556 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 15:00:51.549172 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492426 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 15:00:51.549172 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492428 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 15:00:51.549172 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492431 2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 15:00:51.549172 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492434 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 15:00:51.549172 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492437 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 15:00:51.549172 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492439 2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 15:00:51.549172 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492442 2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 15:00:51.549172 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492444 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 15:00:51.549172 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492447 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 15:00:51.549172 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492450 2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 15:00:51.549172 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492453 2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 15:00:51.549172 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492455 2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 15:00:51.549172 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492458 2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 15:00:51.549172 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492460 2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 15:00:51.549172 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492463 2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 15:00:51.549172 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492466 2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 15:00:51.549989 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492469 2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 15:00:51.549989 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492471 2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 15:00:51.549989 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492474 2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 15:00:51.549989 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492478 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 15:00:51.549989 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492480 2556 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 15:00:51.549989 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492483 2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 15:00:51.549989 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492485 2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 15:00:51.549989 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492488 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 15:00:51.549989 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492490 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 15:00:51.549989 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492492 2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 15:00:51.549989 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492495 2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 15:00:51.549989 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492498 2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 15:00:51.549989 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492500 2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 15:00:51.549989 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492503 2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 15:00:51.549989 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492505 2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 15:00:51.549989 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492509 2556 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 15:00:51.549989 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492511 2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 15:00:51.549989 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492514 2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 15:00:51.549989 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492516 2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 15:00:51.549989 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492518 2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 15:00:51.550748 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492521 2556 feature_gate.go:328] unrecognized feature gate: Example Jul 07 15:00:51.550748 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492524 2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 15:00:51.550748 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492526 2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 15:00:51.550748 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492529 2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 15:00:51.550748 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492531 2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 15:00:51.550748 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492535 2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 15:00:51.550748 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492537 2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 15:00:51.550748 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492540 2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 15:00:51.550748 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492542 2556 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 15:00:51.550748 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492545 2556 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 15:00:51.550748 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492549 2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 15:00:51.550748 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492552 2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 15:00:51.550748 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492555 2556 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 15:00:51.550748 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492558 2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 15:00:51.550748 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492561 2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 15:00:51.550748 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492564 2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 15:00:51.550748 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492567 2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 15:00:51.550748 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492570 2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 15:00:51.550748 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492573 2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 15:00:51.550748 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492575 2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 15:00:51.551357 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492578 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 15:00:51.551357 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492580 2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 15:00:51.551357 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492583 2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 15:00:51.551357 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492585 2556 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 15:00:51.551357 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492588 2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 15:00:51.551357 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492590 2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 15:00:51.551357 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492593 2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 15:00:51.551357 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492595 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 15:00:51.551357 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492598 2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 15:00:51.551357 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492600 2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 15:00:51.551357 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492603 2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 15:00:51.551357 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492607 2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 15:00:51.551357 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492610 2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 15:00:51.551357 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492613 2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 15:00:51.551357 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492616 2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 15:00:51.551357 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492618 2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 15:00:51.551357 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492621 2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 15:00:51.551357 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:51.492625 2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 15:00:51.551804 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.492630 2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 15:00:51.551804 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.492735 2556 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 15:00:51.551804 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.494782 2556 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 15:00:51.551804 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.496194 2556 server.go:1019] "Starting client certificate rotation" Jul 07 15:00:51.551804 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.496286 2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 15:00:51.551804 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.496325 2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 15:00:51.551804 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.498738 2556 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 15:00:51.551804 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.499889 2556 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 15:00:51.551804 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.509710 2556 log.go:25] "Validated CRI v1 runtime API" Jul 07 15:00:51.551804 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.511829 2556 log.go:25] "Validated CRI v1 image API" Jul 07 15:00:51.551804 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.512594 2556 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 15:00:51.551804 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.515155 2556 fs.go:135] Filesystem UUIDs: map[5db9e94f-4e6e-421c-82ae-c39eb9d3f323:/dev/nvme0n1p4 76a3698c-78f4-4a13-b480-46bd75694293:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 07 15:00:51.552158 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.515176 2556 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 15:00:51.552194 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.522803 2556 manager.go:216] Machine: {Timestamp:2026-07-07 15:00:51.52023159 +0000 UTC m=+0.336497664 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099803 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20d4fe307dffda12462cb1c9bb3add SystemUUID:ec20d4fe-307d-ffda-1246-2cb1c9bb3add BootID:8ca86b7c-4f69-4332-93bd-1714d0648f0a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d4:fe:ac:cd:bb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d4:fe:ac:cd:bb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:48:4c:14:da:49 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 15:00:51.552194 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.522954 2556 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 15:00:51.552194 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.523056 2556 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 15:00:51.552194 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.526366 2556 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 15:00:51.552194 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.526426 2556 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-39.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 15:00:51.552194 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.527204 2556 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 15:00:51.552194 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.527216 2556 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 15:00:51.552194 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.527226 2556 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.527854 2556 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.529069 2556 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.530273 2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.531990 2556 state_mem.go:36] "Initialized new in-memory state store" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.532180 2556 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.532216 2556 kubelet.go:486] "Attempting to sync node with API server" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.532230 2556 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.532250 2556 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.532262 2556 kubelet.go:398] "Adding apiserver pod source" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.532278 2556 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.532673 2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.532698 2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.534808 2556 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.536611 2556 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.536638 2556 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.537628 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.537642 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.537648 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.537654 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.537663 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.537670 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.537678 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.537683 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.537689 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.537695 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.537704 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.537709 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.539303 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.539310 2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.542564 2556 server.go:1268] "Started kubelet" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.542642 2556 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.542683 2556 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.542668 2556 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.545156 2556 server.go:310] "Adding debug handlers to kubelet server" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:51.548393 2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.548477 2556 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-39.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.549081 2556 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:51.549630 2556 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 15:00:51.552478 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:51.550278 2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-39.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 15:00:51.553462 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:51.550141 2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-39.ec2.internal.18c00a00edad2514 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-39.ec2.internal,UID:ip-10-0-130-39.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-39.ec2.internal,},FirstTimestamp:2026-07-07 15:00:51.542541588 +0000 UTC m=+0.358807665,LastTimestamp:2026-07-07 15:00:51.542541588 +0000 UTC m=+0.358807665,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-39.ec2.internal,}" Jul 07 15:00:51.553462 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.552542 2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-22fbw" Jul 07 15:00:51.554126 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.554097 2556 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 15:00:51.554126 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.554098 2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 15:00:51.554240 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:51.554208 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-39.ec2.internal\" not found" Jul 07 15:00:51.554751 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.554728 2556 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 15:00:51.555065 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.555050 2556 factory.go:55] Registering systemd factory Jul 07 15:00:51.555152 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.555069 2556 factory.go:223] Registration of the systemd container factory successfully Jul 07 15:00:51.555152 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.555091 2556 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 15:00:51.555152 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.555104 2556 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 15:00:51.555306 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.555158 2556 reconstruct.go:97] "Volume reconstruction finished" Jul 07 15:00:51.555306 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.555166 2556 reconciler.go:29] "Reconciler: start to sync state" Jul 07 15:00:51.555403 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.555377 2556 factory.go:153] Registering CRI-O factory Jul 07 15:00:51.555403 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.555393 2556 factory.go:223] Registration of the crio container factory successfully Jul 07 15:00:51.555502 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.555443 2556 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 15:00:51.555502 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.555465 2556 factory.go:103] Registering Raw factory Jul 07 15:00:51.555502 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.555479 2556 manager.go:1196] Started watching for new ooms in manager Jul 07 15:00:51.556147 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.556131 2556 manager.go:349] Starting recovery of all containers Jul 07 15:00:51.562493 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.562462 2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-22fbw" Jul 07 15:00:51.565778 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:51.565750 2556 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 15:00:51.565911 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:51.565832 2556 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-39.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 15:00:51.566892 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.566878 2556 manager.go:354] Recovery completed Jul 07 15:00:51.570618 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.570603 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:00:51.572944 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.572799 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-39.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:00:51.573022 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.572960 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:00:51.573022 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.572972 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-39.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:00:51.573528 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.573510 2556 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 15:00:51.573528 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.573524 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:00:51.573528 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.573527 2556 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 15:00:51.573676 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.573546 2556 state_mem.go:36] "Initialized new in-memory state store" Jul 07 15:00:51.574569 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.574268 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-39.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:00:51.574569 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.574300 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:00:51.574569 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.574315 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-39.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:00:51.605676 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.605652 2556 policy_none.go:49] "None policy: Start" Jul 07 15:00:51.605676 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.605675 2556 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 15:00:51.605965 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.605691 2556 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 15:00:51.607067 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.607053 2556 policy_none.go:47] "Start" Jul 07 15:00:51.644554 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.644536 2556 manager.go:337] "Starting Device Plugin manager" Jul 07 15:00:51.644652 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:51.644572 2556 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 15:00:51.644652 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.644583 2556 server.go:87] "Starting device plugin registration server" Jul 07 15:00:51.644873 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.644859 2556 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 15:00:51.644963 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.644878 2556 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 15:00:51.645020 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.644961 2556 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 15:00:51.645069 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.645043 2556 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 15:00:51.645069 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.645052 2556 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 15:00:51.645651 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:51.645633 2556 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 15:00:51.645732 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:51.645691 2556 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-39.ec2.internal\" not found" Jul 07 15:00:51.745039 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.745006 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:00:51.746742 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.746714 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-39.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:00:51.746815 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.746757 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:00:51.746815 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.746772 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-39.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:00:51.746815 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.746800 2556 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-39.ec2.internal" Jul 07 15:00:51.753958 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.753922 2556 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-39.ec2.internal" Jul 07 15:00:51.753958 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:51.753952 2556 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-39.ec2.internal\": node \"ip-10-0-130-39.ec2.internal\" not found" Jul 07 15:00:51.770919 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:51.770871 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-39.ec2.internal\" not found" Jul 07 15:00:51.780223 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.780186 2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 15:00:51.781587 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.781568 2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 15:00:51.781658 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.781618 2556 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 15:00:51.781658 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.781646 2556 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 15:00:51.781734 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:51.781691 2556 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 15:00:51.784389 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.784368 2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:00:51.872150 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:51.872119 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-39.ec2.internal\" not found" Jul 07 15:00:51.882011 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.881963 2556 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-39.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-39.ec2.internal"] Jul 07 15:00:51.882087 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.882068 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:00:51.883254 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.883238 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-39.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:00:51.883324 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.883269 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:00:51.883324 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.883280 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-39.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:00:51.884594 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.884581 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:00:51.884737 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.884710 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-39.ec2.internal" Jul 07 15:00:51.884836 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.884754 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:00:51.885424 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.885406 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-39.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:00:51.885529 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.885439 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:00:51.885529 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.885407 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-39.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:00:51.885529 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.885454 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-39.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:00:51.885529 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.885470 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:00:51.885529 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.885482 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-39.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:00:51.886742 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.886722 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-39.ec2.internal" Jul 07 15:00:51.886842 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.886748 2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 15:00:51.887489 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.887471 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-39.ec2.internal" event="NodeHasSufficientMemory" Jul 07 15:00:51.887581 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.887502 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-39.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 15:00:51.887581 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.887518 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-39.ec2.internal" event="NodeHasSufficientPID" Jul 07 15:00:51.905598 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:51.905564 2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-39.ec2.internal\" not found" node="ip-10-0-130-39.ec2.internal" Jul 07 15:00:51.909251 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:51.909231 2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-39.ec2.internal\" not found" node="ip-10-0-130-39.ec2.internal" Jul 07 15:00:51.956257 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.956234 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e1813bc6b6359884c84e2d8d658d0cc0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-39.ec2.internal\" (UID: \"e1813bc6b6359884c84e2d8d658d0cc0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-39.ec2.internal" Jul 07 15:00:51.956413 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.956263 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1813bc6b6359884c84e2d8d658d0cc0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-39.ec2.internal\" (UID: \"e1813bc6b6359884c84e2d8d658d0cc0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-39.ec2.internal" Jul 07 15:00:51.956413 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:51.956289 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1931978687adf475d28ebf31d0441510-config\") pod \"kube-apiserver-proxy-ip-10-0-130-39.ec2.internal\" (UID: \"1931978687adf475d28ebf31d0441510\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-39.ec2.internal" Jul 07 15:00:51.972379 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:51.972352 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-39.ec2.internal\" not found" Jul 07 15:00:52.056998 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:52.056927 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e1813bc6b6359884c84e2d8d658d0cc0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-39.ec2.internal\" (UID: \"e1813bc6b6359884c84e2d8d658d0cc0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-39.ec2.internal" Jul 07 15:00:52.056998 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:52.056980 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1813bc6b6359884c84e2d8d658d0cc0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-39.ec2.internal\" (UID: \"e1813bc6b6359884c84e2d8d658d0cc0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-39.ec2.internal" Jul 07 15:00:52.057136 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:52.057002 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1931978687adf475d28ebf31d0441510-config\") pod \"kube-apiserver-proxy-ip-10-0-130-39.ec2.internal\" (UID: \"1931978687adf475d28ebf31d0441510\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-39.ec2.internal" Jul 07 15:00:52.057136 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:52.057048 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1931978687adf475d28ebf31d0441510-config\") pod \"kube-apiserver-proxy-ip-10-0-130-39.ec2.internal\" (UID: \"1931978687adf475d28ebf31d0441510\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-39.ec2.internal" Jul 07 15:00:52.057136 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:52.057055 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1813bc6b6359884c84e2d8d658d0cc0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-39.ec2.internal\" (UID: \"e1813bc6b6359884c84e2d8d658d0cc0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-39.ec2.internal" Jul 07 15:00:52.057136 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:52.057063 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e1813bc6b6359884c84e2d8d658d0cc0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-39.ec2.internal\" (UID: \"e1813bc6b6359884c84e2d8d658d0cc0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-39.ec2.internal" Jul 07 15:00:52.073225 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:52.073195 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-39.ec2.internal\" not found" Jul 07 15:00:52.173764 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:52.173711 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-39.ec2.internal\" not found" Jul 07 15:00:52.210665 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:52.210640 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-39.ec2.internal" Jul 07 15:00:52.213005 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:52.212984 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-39.ec2.internal" Jul 07 15:00:52.274735 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:52.274693 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-39.ec2.internal\" not found" Jul 07 15:00:52.375761 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:52.375697 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-39.ec2.internal\" not found" Jul 07 15:00:52.476153 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:52.476119 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-39.ec2.internal\" not found" Jul 07 15:00:52.495781 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:52.495747 2556 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 15:00:52.495994 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:52.495967 2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 15:00:52.564266 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:52.564225 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 14:55:52 +0000 UTC" deadline="2027-12-13 19:12:55.149230289 +0000 UTC" Jul 07 15:00:52.564266 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:52.564256 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12580h12m2.584977574s" Jul 07 15:00:52.576769 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:52.576748 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-39.ec2.internal\" not found" Jul 07 15:00:52.648989 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:52.648890 2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:00:52.678130 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:52.678096 2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-39.ec2.internal\" not found" Jul 07 15:00:52.692310 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:52.692284 2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:00:52.718111 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:52.718072 2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:00:52.755134 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:52.755090 2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-39.ec2.internal" Jul 07 15:00:52.767442 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:52.767410 2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 15:00:52.768556 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:52.768534 2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-39.ec2.internal" Jul 07 15:00:52.777818 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:52.777785 2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 15:00:52.831049 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:52.831006 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1813bc6b6359884c84e2d8d658d0cc0.slice/crio-a5d7da88943017cbe2e715b2f009647471ab0e8909c75d5d2dbb9e293e923213 WatchSource:0}: Error finding container a5d7da88943017cbe2e715b2f009647471ab0e8909c75d5d2dbb9e293e923213: Status 404 returned error can't find the container with id a5d7da88943017cbe2e715b2f009647471ab0e8909c75d5d2dbb9e293e923213 Jul 07 15:00:52.831265 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:52.831241 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1931978687adf475d28ebf31d0441510.slice/crio-4257a4ad42a79419bfc6e1eb9cd826406912c130b6c65c1702a45776ed60c213 WatchSource:0}: Error finding container 4257a4ad42a79419bfc6e1eb9cd826406912c130b6c65c1702a45776ed60c213: Status 404 returned error can't find the container with id 4257a4ad42a79419bfc6e1eb9cd826406912c130b6c65c1702a45776ed60c213 Jul 07 15:00:52.835959 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:52.835922 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 15:00:53.533982 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.533891 2556 apiserver.go:52] "Watching apiserver" Jul 07 15:00:53.543474 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.543438 2556 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 15:00:53.545269 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.545222 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-vxlhb","openshift-multus/network-metrics-daemon-dq8k2","openshift-network-diagnostics/network-check-target-hxq5c","openshift-network-operator/iptables-alerter-47cnz","openshift-ovn-kubernetes/ovnkube-node-hqqpl","kube-system/konnectivity-agent-tzhn6","kube-system/kube-apiserver-proxy-ip-10-0-130-39.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm","openshift-cluster-node-tuning-operator/tuned-zs6ml","openshift-dns/node-resolver-spm2h","openshift-image-registry/node-ca-w6w74","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-39.ec2.internal","openshift-multus/multus-additional-cni-plugins-lzp4x","kube-system/global-pull-secret-syncer-n6c4h"] Jul 07 15:00:53.547368 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.547342 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-spm2h" Jul 07 15:00:53.551900 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.550765 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 15:00:53.551900 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.551025 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 15:00:53.551900 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.551212 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qdqnx\"" Jul 07 15:00:53.562248 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.559549 2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 15:00:53.562248 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.560323 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.563170 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.563151 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 15:00:53.563498 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.563405 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 15:00:53.563498 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.563483 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:00:53.563650 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.563432 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 15:00:53.563650 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:53.563568 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxq5c" podUID="a8790322-f857-4238-9294-523d4cf3b5da" Jul 07 15:00:53.563760 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.563412 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wgvzp\"" Jul 07 15:00:53.564344 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.564324 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 15:00:53.565013 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.564593 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 15:00:53.565312 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.565232 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-47cnz" Jul 07 15:00:53.566731 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.566710 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.568558 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.568016 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31976396-202a-47d9-b31f-9bf2efdfd835-tmp-dir\") pod \"node-resolver-spm2h\" (UID: \"31976396-202a-47d9-b31f-9bf2efdfd835\") " pod="openshift-dns/node-resolver-spm2h" Jul 07 15:00:53.568558 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.568057 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a6a7a17-0740-428b-b94c-e073a570c9a7-system-cni-dir\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.568558 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.568086 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a6a7a17-0740-428b-b94c-e073a570c9a7-cnibin\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.568558 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.568111 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0a6a7a17-0740-428b-b94c-e073a570c9a7-os-release\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.568558 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.568138 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6g68\" (UniqueName: \"kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68\") pod \"network-check-target-hxq5c\" (UID: \"a8790322-f857-4238-9294-523d4cf3b5da\") " pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:00:53.568558 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.568165 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s8nk\" (UniqueName: \"kubernetes.io/projected/31976396-202a-47d9-b31f-9bf2efdfd835-kube-api-access-8s8nk\") pod \"node-resolver-spm2h\" (UID: \"31976396-202a-47d9-b31f-9bf2efdfd835\") " pod="openshift-dns/node-resolver-spm2h" Jul 07 15:00:53.568558 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.568190 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a6a7a17-0740-428b-b94c-e073a570c9a7-cni-binary-copy\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.568558 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.568215 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbxtl\" (UniqueName: \"kubernetes.io/projected/0a6a7a17-0740-428b-b94c-e073a570c9a7-kube-api-access-gbxtl\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.568558 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.568242 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/31976396-202a-47d9-b31f-9bf2efdfd835-hosts-file\") pod \"node-resolver-spm2h\" (UID: \"31976396-202a-47d9-b31f-9bf2efdfd835\") " pod="openshift-dns/node-resolver-spm2h" Jul 07 15:00:53.568558 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.568266 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a6a7a17-0740-428b-b94c-e073a570c9a7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.568558 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.568292 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a6a7a17-0740-428b-b94c-e073a570c9a7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.568558 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.568322 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a6a7a17-0740-428b-b94c-e073a570c9a7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.568558 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.568534 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tzhn6" Jul 07 15:00:53.569315 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.569186 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wsfmm\"" Jul 07 15:00:53.570132 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.570112 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.570329 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.570309 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:00:53.570623 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.570602 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 15:00:53.570748 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.570727 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 15:00:53.570879 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.570859 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 15:00:53.571016 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.570994 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rtbnm\"" Jul 07 15:00:53.571241 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.571222 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 15:00:53.571330 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.571279 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 15:00:53.571922 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.571576 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:00:53.571922 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:53.571654 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6c4h" podUID="66af1c92-bb7a-4cda-bb2f-48fa2353b00e" Jul 07 15:00:53.571922 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.571674 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 15:00:53.571922 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.571796 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xkf5b\"" Jul 07 15:00:53.571922 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.571913 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 15:00:53.572229 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.572075 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 15:00:53.572488 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.572471 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 15:00:53.572901 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.572522 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 15:00:53.574030 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.573963 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:00:53.574127 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:53.574037 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dq8k2" podUID="f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a" Jul 07 15:00:53.574770 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.574426 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-b4wdc\"" Jul 07 15:00:53.575134 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.574897 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:00:53.575134 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.575092 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 15:00:53.575405 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.575380 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.576700 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.576519 2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 15:00:53.577636 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.576946 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w6w74" Jul 07 15:00:53.578231 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.578202 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 15:00:53.578347 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.578235 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zw4zt\"" Jul 07 15:00:53.579128 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.578608 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.579705 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.579682 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-j8v28\"" Jul 07 15:00:53.579797 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.579697 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 15:00:53.580132 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.580102 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 15:00:53.581322 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.581062 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 15:00:53.581830 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.581561 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lpxhq\"" Jul 07 15:00:53.581830 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.581611 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 15:00:53.581987 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.581864 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 15:00:53.582082 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.582006 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 15:00:53.600424 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.597138 2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-45fvw" Jul 07 15:00:53.605165 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.605137 2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-45fvw" Jul 07 15:00:53.623913 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.623874 2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 15:00:53.656376 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.656292 2556 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 15:00:53.669191 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669161 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-run-openvswitch\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.669409 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669203 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-ovn-node-metrics-cert\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.669409 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669237 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c17c3a38-de46-4eae-888f-c4424a8e139e-host-slash\") pod \"iptables-alerter-47cnz\" (UID: \"c17c3a38-de46-4eae-888f-c4424a8e139e\") " pod="openshift-network-operator/iptables-alerter-47cnz" Jul 07 15:00:53.669409 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669264 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-sys\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.669409 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669294 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-kubelet-config\") pod \"global-pull-secret-syncer-n6c4h\" (UID: \"66af1c92-bb7a-4cda-bb2f-48fa2353b00e\") " pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:00:53.669409 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669319 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-host-run-ovn-kubernetes\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.669409 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669343 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ce32c7aa-dbf8-4ba2-bdd8-cfe8f4b6ca91-agent-certs\") pod \"konnectivity-agent-tzhn6\" (UID: \"ce32c7aa-dbf8-4ba2-bdd8-cfe8f4b6ca91\") " pod="kube-system/konnectivity-agent-tzhn6" Jul 07 15:00:53.669409 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669368 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/945a2dfd-7ceb-40d4-a093-1357ea0f74a1-device-dir\") pod \"aws-ebs-csi-driver-node-svcgm\" (UID: \"945a2dfd-7ceb-40d4-a093-1357ea0f74a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.669409 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669393 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-etc-sysconfig\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.669841 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669416 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2a62882-861c-46c8-bfd4-d10ffd339dd1-tmp\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.669841 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669440 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-system-cni-dir\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.669841 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669465 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-multus-cni-dir\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.669841 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669488 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-host-slash\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.669841 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669515 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls7z5\" (UniqueName: \"kubernetes.io/projected/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-kube-api-access-ls7z5\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.669841 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669541 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/da182a28-bbef-4bfd-9273-b94f43794676-serviceca\") pod \"node-ca-w6w74\" (UID: \"da182a28-bbef-4bfd-9273-b94f43794676\") " pod="openshift-image-registry/node-ca-w6w74" Jul 07 15:00:53.669841 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669565 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbwfx\" (UniqueName: \"kubernetes.io/projected/945a2dfd-7ceb-40d4-a093-1357ea0f74a1-kube-api-access-hbwfx\") pod \"aws-ebs-csi-driver-node-svcgm\" (UID: \"945a2dfd-7ceb-40d4-a093-1357ea0f74a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.669841 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669589 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c2a62882-861c-46c8-bfd4-d10ffd339dd1-etc-tuned\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.669841 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669618 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-run-ovn\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.669841 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669642 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-multus-socket-dir-parent\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.669841 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669667 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-etc-kubernetes\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.669841 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669700 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-env-overrides\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.669841 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669723 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da182a28-bbef-4bfd-9273-b94f43794676-host\") pod \"node-ca-w6w74\" (UID: \"da182a28-bbef-4bfd-9273-b94f43794676\") " pod="openshift-image-registry/node-ca-w6w74" Jul 07 15:00:53.669841 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669747 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/945a2dfd-7ceb-40d4-a093-1357ea0f74a1-etc-selinux\") pod \"aws-ebs-csi-driver-node-svcgm\" (UID: \"945a2dfd-7ceb-40d4-a093-1357ea0f74a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.670576 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669778 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/945a2dfd-7ceb-40d4-a093-1357ea0f74a1-sys-fs\") pod \"aws-ebs-csi-driver-node-svcgm\" (UID: \"945a2dfd-7ceb-40d4-a093-1357ea0f74a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.670576 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669887 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-etc-systemd\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.670576 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669931 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs\") pod \"network-metrics-daemon-dq8k2\" (UID: \"f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a\") " pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:00:53.670576 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.669993 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-cnibin\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.670576 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670056 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-log-socket\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.670576 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670103 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31976396-202a-47d9-b31f-9bf2efdfd835-tmp-dir\") pod \"node-resolver-spm2h\" (UID: \"31976396-202a-47d9-b31f-9bf2efdfd835\") " pod="openshift-dns/node-resolver-spm2h" Jul 07 15:00:53.670576 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670157 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a6a7a17-0740-428b-b94c-e073a570c9a7-cnibin\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.670576 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670210 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a6a7a17-0740-428b-b94c-e073a570c9a7-cnibin\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.670576 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670215 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0a6a7a17-0740-428b-b94c-e073a570c9a7-os-release\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.670576 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670253 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-node-log\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.670576 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670280 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-etc-kubernetes\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.670576 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670309 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-host-run-k8s-cni-cncf-io\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.670576 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670357 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-ovnkube-script-lib\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.670576 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670361 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0a6a7a17-0740-428b-b94c-e073a570c9a7-os-release\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.671312 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670402 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c17c3a38-de46-4eae-888f-c4424a8e139e-iptables-alerter-script\") pod \"iptables-alerter-47cnz\" (UID: \"c17c3a38-de46-4eae-888f-c4424a8e139e\") " pod="openshift-network-operator/iptables-alerter-47cnz" Jul 07 15:00:53.671312 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670431 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-etc-sysctl-d\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.671312 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670436 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31976396-202a-47d9-b31f-9bf2efdfd835-tmp-dir\") pod \"node-resolver-spm2h\" (UID: \"31976396-202a-47d9-b31f-9bf2efdfd835\") " pod="openshift-dns/node-resolver-spm2h" Jul 07 15:00:53.671312 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670457 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8466910e-c944-4c88-914b-596d470f31a6-cni-binary-copy\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.671312 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670478 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-run\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.671312 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670499 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-lib-modules\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.671312 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670521 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-host-kubelet\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.671312 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670544 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8466910e-c944-4c88-914b-596d470f31a6-multus-daemon-config\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.671312 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670568 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dms9h\" (UniqueName: \"kubernetes.io/projected/8466910e-c944-4c88-914b-596d470f31a6-kube-api-access-dms9h\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.671312 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670599 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6g68\" (UniqueName: \"kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68\") pod \"network-check-target-hxq5c\" (UID: \"a8790322-f857-4238-9294-523d4cf3b5da\") " pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:00:53.671312 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670638 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s8nk\" (UniqueName: \"kubernetes.io/projected/31976396-202a-47d9-b31f-9bf2efdfd835-kube-api-access-8s8nk\") pod \"node-resolver-spm2h\" (UID: \"31976396-202a-47d9-b31f-9bf2efdfd835\") " pod="openshift-dns/node-resolver-spm2h" Jul 07 15:00:53.671312 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670663 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fn8v\" (UniqueName: \"kubernetes.io/projected/c2a62882-861c-46c8-bfd4-d10ffd339dd1-kube-api-access-5fn8v\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.671312 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670687 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-host-cni-bin\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.671312 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670715 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-host-cni-netd\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.672261 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670738 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-dbus\") pod \"global-pull-secret-syncer-n6c4h\" (UID: \"66af1c92-bb7a-4cda-bb2f-48fa2353b00e\") " pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:00:53.672261 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670763 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-host-var-lib-cni-multus\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.672261 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670787 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-hostroot\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.672261 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670810 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-host-run-multus-certs\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.672261 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670837 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/945a2dfd-7ceb-40d4-a093-1357ea0f74a1-socket-dir\") pod \"aws-ebs-csi-driver-node-svcgm\" (UID: \"945a2dfd-7ceb-40d4-a093-1357ea0f74a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.672261 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670865 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a6a7a17-0740-428b-b94c-e073a570c9a7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.672261 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670919 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-host-var-lib-cni-bin\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.672261 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670954 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-host-var-lib-kubelet\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.672261 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.670981 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-var-lib-openvswitch\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.672261 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671025 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hxhk\" (UniqueName: \"kubernetes.io/projected/da182a28-bbef-4bfd-9273-b94f43794676-kube-api-access-6hxhk\") pod \"node-ca-w6w74\" (UID: \"da182a28-bbef-4bfd-9273-b94f43794676\") " pod="openshift-image-registry/node-ca-w6w74" Jul 07 15:00:53.672261 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671051 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/945a2dfd-7ceb-40d4-a093-1357ea0f74a1-registration-dir\") pod \"aws-ebs-csi-driver-node-svcgm\" (UID: \"945a2dfd-7ceb-40d4-a093-1357ea0f74a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.672261 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671068 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a6a7a17-0740-428b-b94c-e073a570c9a7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.672261 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671082 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a6a7a17-0740-428b-b94c-e073a570c9a7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.672261 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671127 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-var-lib-kubelet\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.673003 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671249 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.673003 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671278 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-ovnkube-config\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.673003 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671311 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/945a2dfd-7ceb-40d4-a093-1357ea0f74a1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-svcgm\" (UID: \"945a2dfd-7ceb-40d4-a093-1357ea0f74a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.673003 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671335 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-etc-modprobe-d\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.673003 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671360 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-etc-sysctl-conf\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.673003 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671392 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a6a7a17-0740-428b-b94c-e073a570c9a7-system-cni-dir\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.673003 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671438 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-multus-conf-dir\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.673003 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671485 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ce32c7aa-dbf8-4ba2-bdd8-cfe8f4b6ca91-konnectivity-ca\") pod \"konnectivity-agent-tzhn6\" (UID: \"ce32c7aa-dbf8-4ba2-bdd8-cfe8f4b6ca91\") " pod="kube-system/konnectivity-agent-tzhn6" Jul 07 15:00:53.673003 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671520 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a6a7a17-0740-428b-b94c-e073a570c9a7-cni-binary-copy\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.673003 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671546 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbxtl\" (UniqueName: \"kubernetes.io/projected/0a6a7a17-0740-428b-b94c-e073a570c9a7-kube-api-access-gbxtl\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.673003 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671572 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-host\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.673003 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671598 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb7qt\" (UniqueName: \"kubernetes.io/projected/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-kube-api-access-bb7qt\") pod \"network-metrics-daemon-dq8k2\" (UID: \"f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a\") " pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:00:53.673003 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671623 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-host-run-netns\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.673812 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671648 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-systemd-units\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.673812 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671737 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-host-run-netns\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.673812 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671751 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a6a7a17-0740-428b-b94c-e073a570c9a7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.673812 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671769 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-etc-openvswitch\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.673812 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671797 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/31976396-202a-47d9-b31f-9bf2efdfd835-hosts-file\") pod \"node-resolver-spm2h\" (UID: \"31976396-202a-47d9-b31f-9bf2efdfd835\") " pod="openshift-dns/node-resolver-spm2h" Jul 07 15:00:53.673812 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671814 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a6a7a17-0740-428b-b94c-e073a570c9a7-system-cni-dir\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.673812 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671824 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a6a7a17-0740-428b-b94c-e073a570c9a7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.673812 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671852 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twx6z\" (UniqueName: \"kubernetes.io/projected/c17c3a38-de46-4eae-888f-c4424a8e139e-kube-api-access-twx6z\") pod \"iptables-alerter-47cnz\" (UID: \"c17c3a38-de46-4eae-888f-c4424a8e139e\") " pod="openshift-network-operator/iptables-alerter-47cnz" Jul 07 15:00:53.673812 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671878 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret\") pod \"global-pull-secret-syncer-n6c4h\" (UID: \"66af1c92-bb7a-4cda-bb2f-48fa2353b00e\") " pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:00:53.673812 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.671914 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-os-release\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.673812 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.672014 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-run-systemd\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.673812 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.672127 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/31976396-202a-47d9-b31f-9bf2efdfd835-hosts-file\") pod \"node-resolver-spm2h\" (UID: \"31976396-202a-47d9-b31f-9bf2efdfd835\") " pod="openshift-dns/node-resolver-spm2h" Jul 07 15:00:53.673812 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.672289 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a6a7a17-0740-428b-b94c-e073a570c9a7-cni-binary-copy\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.673812 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.672558 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a6a7a17-0740-428b-b94c-e073a570c9a7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.678277 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.678031 2556 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 15:00:53.678277 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:53.678043 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:00:53.678277 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:53.678143 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:00:53.678277 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:53.678166 2556 projected.go:196] Error preparing data for projected volume kube-api-access-l6g68 for pod openshift-network-diagnostics/network-check-target-hxq5c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:00:53.678277 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:53.678241 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68 podName:a8790322-f857-4238-9294-523d4cf3b5da nodeName:}" failed. No retries permitted until 2026-07-07 15:00:54.178208398 +0000 UTC m=+2.994474423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-l6g68" (UniqueName: "kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68") pod "network-check-target-hxq5c" (UID: "a8790322-f857-4238-9294-523d4cf3b5da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:00:53.682967 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.682912 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbxtl\" (UniqueName: \"kubernetes.io/projected/0a6a7a17-0740-428b-b94c-e073a570c9a7-kube-api-access-gbxtl\") pod \"multus-additional-cni-plugins-lzp4x\" (UID: \"0a6a7a17-0740-428b-b94c-e073a570c9a7\") " pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.682967 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.682923 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s8nk\" (UniqueName: \"kubernetes.io/projected/31976396-202a-47d9-b31f-9bf2efdfd835-kube-api-access-8s8nk\") pod \"node-resolver-spm2h\" (UID: \"31976396-202a-47d9-b31f-9bf2efdfd835\") " pod="openshift-dns/node-resolver-spm2h" Jul 07 15:00:53.772279 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772237 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-run\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.772279 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772285 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-lib-modules\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.772535 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772311 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-host-kubelet\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.772535 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772339 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8466910e-c944-4c88-914b-596d470f31a6-multus-daemon-config\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.772535 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772382 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dms9h\" (UniqueName: \"kubernetes.io/projected/8466910e-c944-4c88-914b-596d470f31a6-kube-api-access-dms9h\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.772535 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772389 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-run\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.772535 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772423 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fn8v\" (UniqueName: \"kubernetes.io/projected/c2a62882-861c-46c8-bfd4-d10ffd339dd1-kube-api-access-5fn8v\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.772535 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772445 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-host-cni-bin\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.772535 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772471 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-host-cni-netd\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.772535 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772499 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-lib-modules\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.772535 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772496 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-dbus\") pod \"global-pull-secret-syncer-n6c4h\" (UID: \"66af1c92-bb7a-4cda-bb2f-48fa2353b00e\") " pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:00:53.773052 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772545 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-host-var-lib-cni-multus\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.773052 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772570 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-hostroot\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.773052 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772593 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-host-run-multus-certs\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.773052 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772616 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/945a2dfd-7ceb-40d4-a093-1357ea0f74a1-socket-dir\") pod \"aws-ebs-csi-driver-node-svcgm\" (UID: \"945a2dfd-7ceb-40d4-a093-1357ea0f74a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.773052 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772639 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-host-var-lib-cni-bin\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.773052 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772659 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-host-var-lib-kubelet\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.773052 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772684 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-var-lib-openvswitch\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.773052 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772692 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-dbus\") pod \"global-pull-secret-syncer-n6c4h\" (UID: \"66af1c92-bb7a-4cda-bb2f-48fa2353b00e\") " pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:00:53.773052 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772709 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hxhk\" (UniqueName: \"kubernetes.io/projected/da182a28-bbef-4bfd-9273-b94f43794676-kube-api-access-6hxhk\") pod \"node-ca-w6w74\" (UID: \"da182a28-bbef-4bfd-9273-b94f43794676\") " pod="openshift-image-registry/node-ca-w6w74" Jul 07 15:00:53.773052 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772736 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/945a2dfd-7ceb-40d4-a093-1357ea0f74a1-registration-dir\") pod \"aws-ebs-csi-driver-node-svcgm\" (UID: \"945a2dfd-7ceb-40d4-a093-1357ea0f74a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.773052 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772760 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-var-lib-kubelet\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.773052 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772796 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.773052 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772822 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-ovnkube-config\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.773052 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772846 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/945a2dfd-7ceb-40d4-a093-1357ea0f74a1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-svcgm\" (UID: \"945a2dfd-7ceb-40d4-a093-1357ea0f74a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.773823 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772870 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-etc-modprobe-d\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.773823 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772893 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-etc-sysctl-conf\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.773823 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772941 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-multus-conf-dir\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.773823 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.772968 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ce32c7aa-dbf8-4ba2-bdd8-cfe8f4b6ca91-konnectivity-ca\") pod \"konnectivity-agent-tzhn6\" (UID: \"ce32c7aa-dbf8-4ba2-bdd8-cfe8f4b6ca91\") " pod="kube-system/konnectivity-agent-tzhn6" Jul 07 15:00:53.773823 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773005 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-host\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.773823 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773037 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb7qt\" (UniqueName: \"kubernetes.io/projected/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-kube-api-access-bb7qt\") pod \"network-metrics-daemon-dq8k2\" (UID: \"f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a\") " pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:00:53.773823 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773078 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-host-run-netns\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.773823 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773107 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-systemd-units\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.773823 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773132 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-host-run-netns\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.773823 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773135 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8466910e-c944-4c88-914b-596d470f31a6-multus-daemon-config\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.773823 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773182 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-host-cni-bin\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.773823 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773205 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-host-run-netns\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.773823 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773221 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-host-run-multus-certs\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.773823 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773229 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-hostroot\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.773823 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773186 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-host-var-lib-cni-multus\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.774572 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773284 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/945a2dfd-7ceb-40d4-a093-1357ea0f74a1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-svcgm\" (UID: \"945a2dfd-7ceb-40d4-a093-1357ea0f74a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.774572 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773326 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-host-cni-netd\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.774572 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773338 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/945a2dfd-7ceb-40d4-a093-1357ea0f74a1-socket-dir\") pod \"aws-ebs-csi-driver-node-svcgm\" (UID: \"945a2dfd-7ceb-40d4-a093-1357ea0f74a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.774572 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773356 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-etc-openvswitch\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.774572 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773391 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twx6z\" (UniqueName: \"kubernetes.io/projected/c17c3a38-de46-4eae-888f-c4424a8e139e-kube-api-access-twx6z\") pod \"iptables-alerter-47cnz\" (UID: \"c17c3a38-de46-4eae-888f-c4424a8e139e\") " pod="openshift-network-operator/iptables-alerter-47cnz" Jul 07 15:00:53.774572 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773419 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret\") pod \"global-pull-secret-syncer-n6c4h\" (UID: \"66af1c92-bb7a-4cda-bb2f-48fa2353b00e\") " pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:00:53.774572 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773446 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-os-release\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.774572 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773450 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-host-var-lib-cni-bin\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.774572 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773472 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-run-systemd\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.774572 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773492 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-host-var-lib-kubelet\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.774572 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773503 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-run-openvswitch\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.774572 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773530 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-ovn-node-metrics-cert\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.774572 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773559 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c17c3a38-de46-4eae-888f-c4424a8e139e-host-slash\") pod \"iptables-alerter-47cnz\" (UID: \"c17c3a38-de46-4eae-888f-c4424a8e139e\") " pod="openshift-network-operator/iptables-alerter-47cnz" Jul 07 15:00:53.774572 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773563 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-var-lib-openvswitch\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.775346 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773587 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-systemd-units\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.775346 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773585 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-sys\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.775346 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773630 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-host-kubelet\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.775346 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773654 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-kubelet-config\") pod \"global-pull-secret-syncer-n6c4h\" (UID: \"66af1c92-bb7a-4cda-bb2f-48fa2353b00e\") " pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:00:53.775346 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773682 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-host-run-ovn-kubernetes\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.775346 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773690 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/945a2dfd-7ceb-40d4-a093-1357ea0f74a1-registration-dir\") pod \"aws-ebs-csi-driver-node-svcgm\" (UID: \"945a2dfd-7ceb-40d4-a093-1357ea0f74a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.775346 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773708 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ce32c7aa-dbf8-4ba2-bdd8-cfe8f4b6ca91-agent-certs\") pod \"konnectivity-agent-tzhn6\" (UID: \"ce32c7aa-dbf8-4ba2-bdd8-cfe8f4b6ca91\") " pod="kube-system/konnectivity-agent-tzhn6" Jul 07 15:00:53.775346 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773735 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/945a2dfd-7ceb-40d4-a093-1357ea0f74a1-device-dir\") pod \"aws-ebs-csi-driver-node-svcgm\" (UID: \"945a2dfd-7ceb-40d4-a093-1357ea0f74a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.775346 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773742 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-var-lib-kubelet\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.775346 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773765 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-etc-sysconfig\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.775346 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773782 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.775346 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773792 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2a62882-861c-46c8-bfd4-d10ffd339dd1-tmp\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.775346 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773859 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-etc-sysctl-conf\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.775346 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773532 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-multus-conf-dir\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.776118 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773913 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-run-systemd\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.776118 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773530 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-host\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.776118 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774194 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-run-openvswitch\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.776118 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774445 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-ovnkube-config\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.776118 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774445 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ce32c7aa-dbf8-4ba2-bdd8-cfe8f4b6ca91-konnectivity-ca\") pod \"konnectivity-agent-tzhn6\" (UID: \"ce32c7aa-dbf8-4ba2-bdd8-cfe8f4b6ca91\") " pod="kube-system/konnectivity-agent-tzhn6" Jul 07 15:00:53.776118 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774491 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-system-cni-dir\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.776118 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774508 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-host-run-netns\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.776118 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774534 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-multus-cni-dir\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.776118 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774560 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-host-slash\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.776118 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774566 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c17c3a38-de46-4eae-888f-c4424a8e139e-host-slash\") pod \"iptables-alerter-47cnz\" (UID: \"c17c3a38-de46-4eae-888f-c4424a8e139e\") " pod="openshift-network-operator/iptables-alerter-47cnz" Jul 07 15:00:53.776118 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774596 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls7z5\" (UniqueName: \"kubernetes.io/projected/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-kube-api-access-ls7z5\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.776118 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774603 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-host-slash\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.776118 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774623 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/da182a28-bbef-4bfd-9273-b94f43794676-serviceca\") pod \"node-ca-w6w74\" (UID: \"da182a28-bbef-4bfd-9273-b94f43794676\") " pod="openshift-image-registry/node-ca-w6w74" Jul 07 15:00:53.776118 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774650 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbwfx\" (UniqueName: \"kubernetes.io/projected/945a2dfd-7ceb-40d4-a093-1357ea0f74a1-kube-api-access-hbwfx\") pod \"aws-ebs-csi-driver-node-svcgm\" (UID: \"945a2dfd-7ceb-40d4-a093-1357ea0f74a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.776118 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774653 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-etc-openvswitch\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.776832 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774677 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c2a62882-861c-46c8-bfd4-d10ffd339dd1-etc-tuned\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.776832 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774703 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-run-ovn\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.776832 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774727 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-multus-socket-dir-parent\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.776832 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:53.774729 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:00:53.776832 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774781 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-os-release\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.776832 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:53.774800 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret podName:66af1c92-bb7a-4cda-bb2f-48fa2353b00e nodeName:}" failed. No retries permitted until 2026-07-07 15:00:54.274782458 +0000 UTC m=+3.091048477 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret") pod "global-pull-secret-syncer-n6c4h" (UID: "66af1c92-bb7a-4cda-bb2f-48fa2353b00e") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:00:53.776832 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774801 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-kubelet-config\") pod \"global-pull-secret-syncer-n6c4h\" (UID: \"66af1c92-bb7a-4cda-bb2f-48fa2353b00e\") " pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:00:53.776832 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774837 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-sys\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.776832 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.773446 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-etc-modprobe-d\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.776832 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774852 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-etc-kubernetes\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.776832 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774894 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-system-cni-dir\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.776832 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774899 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-env-overrides\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.776832 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774930 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da182a28-bbef-4bfd-9273-b94f43794676-host\") pod \"node-ca-w6w74\" (UID: \"da182a28-bbef-4bfd-9273-b94f43794676\") " pod="openshift-image-registry/node-ca-w6w74" Jul 07 15:00:53.776832 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774961 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/945a2dfd-7ceb-40d4-a093-1357ea0f74a1-etc-selinux\") pod \"aws-ebs-csi-driver-node-svcgm\" (UID: \"945a2dfd-7ceb-40d4-a093-1357ea0f74a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.776832 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774967 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-multus-cni-dir\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.777462 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.774990 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/945a2dfd-7ceb-40d4-a093-1357ea0f74a1-sys-fs\") pod \"aws-ebs-csi-driver-node-svcgm\" (UID: \"945a2dfd-7ceb-40d4-a093-1357ea0f74a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.777462 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.775015 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-host-run-ovn-kubernetes\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.777462 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.775021 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-etc-systemd\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.777462 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.775071 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-etc-kubernetes\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.777462 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.775429 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/da182a28-bbef-4bfd-9273-b94f43794676-serviceca\") pod \"node-ca-w6w74\" (UID: \"da182a28-bbef-4bfd-9273-b94f43794676\") " pod="openshift-image-registry/node-ca-w6w74" Jul 07 15:00:53.777462 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.775455 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/945a2dfd-7ceb-40d4-a093-1357ea0f74a1-etc-selinux\") pod \"aws-ebs-csi-driver-node-svcgm\" (UID: \"945a2dfd-7ceb-40d4-a093-1357ea0f74a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.777462 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.775493 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-env-overrides\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.777462 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.775512 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/da182a28-bbef-4bfd-9273-b94f43794676-host\") pod \"node-ca-w6w74\" (UID: \"da182a28-bbef-4bfd-9273-b94f43794676\") " pod="openshift-image-registry/node-ca-w6w74" Jul 07 15:00:53.777462 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.775555 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-run-ovn\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.777462 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.775567 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/945a2dfd-7ceb-40d4-a093-1357ea0f74a1-sys-fs\") pod \"aws-ebs-csi-driver-node-svcgm\" (UID: \"945a2dfd-7ceb-40d4-a093-1357ea0f74a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.777462 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.775625 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-etc-systemd\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.777462 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.775677 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-multus-socket-dir-parent\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.777462 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.775738 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-etc-sysconfig\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.777462 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.775747 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/945a2dfd-7ceb-40d4-a093-1357ea0f74a1-device-dir\") pod \"aws-ebs-csi-driver-node-svcgm\" (UID: \"945a2dfd-7ceb-40d4-a093-1357ea0f74a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.777462 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.775851 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs\") pod \"network-metrics-daemon-dq8k2\" (UID: \"f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a\") " pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:00:53.778147 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.775901 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-cnibin\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.778147 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.775928 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-log-socket\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.778147 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.775964 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-node-log\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.778147 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.775993 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-etc-kubernetes\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.778147 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:53.776005 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:00:53.778147 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.776020 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-host-run-k8s-cni-cncf-io\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.778147 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:53.776059 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs podName:f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a nodeName:}" failed. No retries permitted until 2026-07-07 15:00:54.276043497 +0000 UTC m=+3.092309534 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs") pod "network-metrics-daemon-dq8k2" (UID: "f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:00:53.778147 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.776086 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-host-run-k8s-cni-cncf-io\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.778147 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.776083 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-ovnkube-script-lib\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.778147 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.776131 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c17c3a38-de46-4eae-888f-c4424a8e139e-iptables-alerter-script\") pod \"iptables-alerter-47cnz\" (UID: \"c17c3a38-de46-4eae-888f-c4424a8e139e\") " pod="openshift-network-operator/iptables-alerter-47cnz" Jul 07 15:00:53.778147 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.776645 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-ovnkube-script-lib\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.778147 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.776712 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2a62882-861c-46c8-bfd4-d10ffd339dd1-tmp\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.778147 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.776718 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-etc-sysctl-d\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.778147 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.776786 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8466910e-c944-4c88-914b-596d470f31a6-cni-binary-copy\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.778791 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.776820 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-etc-sysctl-d\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.778791 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.776876 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-log-socket\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.778791 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.776938 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8466910e-c944-4c88-914b-596d470f31a6-cnibin\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.778791 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.777011 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-node-log\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.778791 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.777019 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2a62882-861c-46c8-bfd4-d10ffd339dd1-etc-kubernetes\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.778791 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.777435 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8466910e-c944-4c88-914b-596d470f31a6-cni-binary-copy\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.778791 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.778140 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-ovn-node-metrics-cert\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.778791 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.778176 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c17c3a38-de46-4eae-888f-c4424a8e139e-iptables-alerter-script\") pod \"iptables-alerter-47cnz\" (UID: \"c17c3a38-de46-4eae-888f-c4424a8e139e\") " pod="openshift-network-operator/iptables-alerter-47cnz" Jul 07 15:00:53.779171 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.779041 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c2a62882-861c-46c8-bfd4-d10ffd339dd1-etc-tuned\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.779833 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.779808 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ce32c7aa-dbf8-4ba2-bdd8-cfe8f4b6ca91-agent-certs\") pod \"konnectivity-agent-tzhn6\" (UID: \"ce32c7aa-dbf8-4ba2-bdd8-cfe8f4b6ca91\") " pod="kube-system/konnectivity-agent-tzhn6" Jul 07 15:00:53.787832 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.787766 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dms9h\" (UniqueName: \"kubernetes.io/projected/8466910e-c944-4c88-914b-596d470f31a6-kube-api-access-dms9h\") pod \"multus-vxlhb\" (UID: \"8466910e-c944-4c88-914b-596d470f31a6\") " pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.788601 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.788576 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twx6z\" (UniqueName: \"kubernetes.io/projected/c17c3a38-de46-4eae-888f-c4424a8e139e-kube-api-access-twx6z\") pod \"iptables-alerter-47cnz\" (UID: \"c17c3a38-de46-4eae-888f-c4424a8e139e\") " pod="openshift-network-operator/iptables-alerter-47cnz" Jul 07 15:00:53.788840 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.788817 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fn8v\" (UniqueName: \"kubernetes.io/projected/c2a62882-861c-46c8-bfd4-d10ffd339dd1-kube-api-access-5fn8v\") pod \"tuned-zs6ml\" (UID: \"c2a62882-861c-46c8-bfd4-d10ffd339dd1\") " pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.789711 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.789645 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb7qt\" (UniqueName: \"kubernetes.io/projected/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-kube-api-access-bb7qt\") pod \"network-metrics-daemon-dq8k2\" (UID: \"f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a\") " pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:00:53.789808 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.789706 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-39.ec2.internal" event={"ID":"e1813bc6b6359884c84e2d8d658d0cc0","Type":"ContainerStarted","Data":"a5d7da88943017cbe2e715b2f009647471ab0e8909c75d5d2dbb9e293e923213"} Jul 07 15:00:53.790445 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.790420 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hxhk\" (UniqueName: \"kubernetes.io/projected/da182a28-bbef-4bfd-9273-b94f43794676-kube-api-access-6hxhk\") pod \"node-ca-w6w74\" (UID: \"da182a28-bbef-4bfd-9273-b94f43794676\") " pod="openshift-image-registry/node-ca-w6w74" Jul 07 15:00:53.790820 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.790773 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbwfx\" (UniqueName: \"kubernetes.io/projected/945a2dfd-7ceb-40d4-a093-1357ea0f74a1-kube-api-access-hbwfx\") pod \"aws-ebs-csi-driver-node-svcgm\" (UID: \"945a2dfd-7ceb-40d4-a093-1357ea0f74a1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.791223 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.791196 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-39.ec2.internal" event={"ID":"1931978687adf475d28ebf31d0441510","Type":"ContainerStarted","Data":"4257a4ad42a79419bfc6e1eb9cd826406912c130b6c65c1702a45776ed60c213"} Jul 07 15:00:53.792091 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.792063 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls7z5\" (UniqueName: \"kubernetes.io/projected/3e8f59d5-d4a6-4685-a7f3-ff556ff165a5-kube-api-access-ls7z5\") pod \"ovnkube-node-hqqpl\" (UID: \"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.864511 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.864471 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-spm2h" Jul 07 15:00:53.875264 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:53.875235 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31976396_202a_47d9_b31f_9bf2efdfd835.slice/crio-f375c2008d754149ef38e4dfd817fc842df060e3216966e347cc30f0fbc02f96 WatchSource:0}: Error finding container f375c2008d754149ef38e4dfd817fc842df060e3216966e347cc30f0fbc02f96: Status 404 returned error can't find the container with id f375c2008d754149ef38e4dfd817fc842df060e3216966e347cc30f0fbc02f96 Jul 07 15:00:53.877172 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.876621 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lzp4x" Jul 07 15:00:53.887267 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:53.887236 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a6a7a17_0740_428b_b94c_e073a570c9a7.slice/crio-b15e109e3f0aed56c820fade79fb8ff4866aa5b707f5b59a774e23106366f420 WatchSource:0}: Error finding container b15e109e3f0aed56c820fade79fb8ff4866aa5b707f5b59a774e23106366f420: Status 404 returned error can't find the container with id b15e109e3f0aed56c820fade79fb8ff4866aa5b707f5b59a774e23106366f420 Jul 07 15:00:53.889070 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.889049 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-47cnz" Jul 07 15:00:53.897859 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.897820 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:00:53.898122 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:53.898093 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc17c3a38_de46_4eae_888f_c4424a8e139e.slice/crio-906b9f041f3a6328367f2a440b5342d2b29ad22ab59f260811670a0e52906c66 WatchSource:0}: Error finding container 906b9f041f3a6328367f2a440b5342d2b29ad22ab59f260811670a0e52906c66: Status 404 returned error can't find the container with id 906b9f041f3a6328367f2a440b5342d2b29ad22ab59f260811670a0e52906c66 Jul 07 15:00:53.907337 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.907311 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tzhn6" Jul 07 15:00:53.907520 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:53.907496 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e8f59d5_d4a6_4685_a7f3_ff556ff165a5.slice/crio-e4d7d3a6789b0064e3372d62a17115410a057e7764fd1fe2c1d2b1b7dbc407f0 WatchSource:0}: Error finding container e4d7d3a6789b0064e3372d62a17115410a057e7764fd1fe2c1d2b1b7dbc407f0: Status 404 returned error can't find the container with id e4d7d3a6789b0064e3372d62a17115410a057e7764fd1fe2c1d2b1b7dbc407f0 Jul 07 15:00:53.916252 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.916227 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" Jul 07 15:00:53.926372 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.926295 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vxlhb" Jul 07 15:00:53.928158 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:53.928130 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2a62882_861c_46c8_bfd4_d10ffd339dd1.slice/crio-e536d00c3966c4f8b0e2b42f08a569459995dd9cbd8d56b564895dc1af7a576c WatchSource:0}: Error finding container e536d00c3966c4f8b0e2b42f08a569459995dd9cbd8d56b564895dc1af7a576c: Status 404 returned error can't find the container with id e536d00c3966c4f8b0e2b42f08a569459995dd9cbd8d56b564895dc1af7a576c Jul 07 15:00:53.936494 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.936401 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-w6w74" Jul 07 15:00:53.943332 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:53.943307 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" Jul 07 15:00:53.945871 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:53.945849 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda182a28_bbef_4bfd_9273_b94f43794676.slice/crio-7a5886f02b13380b99e76aadca567bcdc1c64802b228cc9553d191336563d6c9 WatchSource:0}: Error finding container 7a5886f02b13380b99e76aadca567bcdc1c64802b228cc9553d191336563d6c9: Status 404 returned error can't find the container with id 7a5886f02b13380b99e76aadca567bcdc1c64802b228cc9553d191336563d6c9 Jul 07 15:00:54.179912 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:54.179799 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6g68\" (UniqueName: \"kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68\") pod \"network-check-target-hxq5c\" (UID: \"a8790322-f857-4238-9294-523d4cf3b5da\") " pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:00:54.180081 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:54.179972 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:00:54.180081 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:54.179997 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:00:54.180081 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:54.180011 2556 projected.go:196] Error preparing data for projected volume kube-api-access-l6g68 for pod openshift-network-diagnostics/network-check-target-hxq5c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:00:54.180081 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:54.180082 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68 podName:a8790322-f857-4238-9294-523d4cf3b5da nodeName:}" failed. No retries permitted until 2026-07-07 15:00:55.180064732 +0000 UTC m=+3.996330750 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-l6g68" (UniqueName: "kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68") pod "network-check-target-hxq5c" (UID: "a8790322-f857-4238-9294-523d4cf3b5da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:00:54.280919 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:54.280874 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret\") pod \"global-pull-secret-syncer-n6c4h\" (UID: \"66af1c92-bb7a-4cda-bb2f-48fa2353b00e\") " pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:00:54.281103 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:54.280955 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs\") pod \"network-metrics-daemon-dq8k2\" (UID: \"f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a\") " pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:00:54.281103 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:54.281061 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:00:54.281211 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:54.281132 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret podName:66af1c92-bb7a-4cda-bb2f-48fa2353b00e nodeName:}" failed. No retries permitted until 2026-07-07 15:00:55.28111713 +0000 UTC m=+4.097383152 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret") pod "global-pull-secret-syncer-n6c4h" (UID: "66af1c92-bb7a-4cda-bb2f-48fa2353b00e") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:00:54.281211 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:54.281061 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:00:54.281211 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:54.281202 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs podName:f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a nodeName:}" failed. No retries permitted until 2026-07-07 15:00:55.281187294 +0000 UTC m=+4.097453316 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs") pod "network-metrics-daemon-dq8k2" (UID: "f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:00:54.607886 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:54.607762 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 14:55:54 +0000 UTC" deadline="2028-02-14 14:00:51.863498781 +0000 UTC" Jul 07 15:00:54.607886 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:54.607807 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14086h59m57.255697083s" Jul 07 15:00:54.795005 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:54.794951 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w6w74" event={"ID":"da182a28-bbef-4bfd-9273-b94f43794676","Type":"ContainerStarted","Data":"7a5886f02b13380b99e76aadca567bcdc1c64802b228cc9553d191336563d6c9"} Jul 07 15:00:54.796585 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:54.796559 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vxlhb" event={"ID":"8466910e-c944-4c88-914b-596d470f31a6","Type":"ContainerStarted","Data":"551121bb9502bcb78b94a8ede749b01c13719e145c0c78c3534e46ca957e061e"} Jul 07 15:00:54.798114 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:54.798055 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-47cnz" event={"ID":"c17c3a38-de46-4eae-888f-c4424a8e139e","Type":"ContainerStarted","Data":"906b9f041f3a6328367f2a440b5342d2b29ad22ab59f260811670a0e52906c66"} Jul 07 15:00:54.799473 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:54.799443 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lzp4x" event={"ID":"0a6a7a17-0740-428b-b94c-e073a570c9a7","Type":"ContainerStarted","Data":"b15e109e3f0aed56c820fade79fb8ff4866aa5b707f5b59a774e23106366f420"} Jul 07 15:00:54.800840 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:54.800812 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-spm2h" event={"ID":"31976396-202a-47d9-b31f-9bf2efdfd835","Type":"ContainerStarted","Data":"f375c2008d754149ef38e4dfd817fc842df060e3216966e347cc30f0fbc02f96"} Jul 07 15:00:54.802208 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:54.802181 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" event={"ID":"c2a62882-861c-46c8-bfd4-d10ffd339dd1","Type":"ContainerStarted","Data":"e536d00c3966c4f8b0e2b42f08a569459995dd9cbd8d56b564895dc1af7a576c"} Jul 07 15:00:54.803587 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:54.803557 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tzhn6" event={"ID":"ce32c7aa-dbf8-4ba2-bdd8-cfe8f4b6ca91","Type":"ContainerStarted","Data":"e1c86d18945183baaa35c5e3350e3780857af3af810b8174209494ba0c67a2f5"} Jul 07 15:00:54.804937 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:54.804898 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" event={"ID":"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5","Type":"ContainerStarted","Data":"e4d7d3a6789b0064e3372d62a17115410a057e7764fd1fe2c1d2b1b7dbc407f0"} Jul 07 15:00:55.064814 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:00:55.064787 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod945a2dfd_7ceb_40d4_a093_1357ea0f74a1.slice/crio-67156fdd00360a24f34c78b6480edd0286386f715e4ff2f3749a9a7275585342 WatchSource:0}: Error finding container 67156fdd00360a24f34c78b6480edd0286386f715e4ff2f3749a9a7275585342: Status 404 returned error can't find the container with id 67156fdd00360a24f34c78b6480edd0286386f715e4ff2f3749a9a7275585342 Jul 07 15:00:55.189659 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:55.189267 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6g68\" (UniqueName: \"kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68\") pod \"network-check-target-hxq5c\" (UID: \"a8790322-f857-4238-9294-523d4cf3b5da\") " pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:00:55.189659 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:55.189542 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:00:55.189659 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:55.189566 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:00:55.189659 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:55.189579 2556 projected.go:196] Error preparing data for projected volume kube-api-access-l6g68 for pod openshift-network-diagnostics/network-check-target-hxq5c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:00:55.189659 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:55.189659 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68 podName:a8790322-f857-4238-9294-523d4cf3b5da nodeName:}" failed. No retries permitted until 2026-07-07 15:00:57.18962043 +0000 UTC m=+6.005886471 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-l6g68" (UniqueName: "kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68") pod "network-check-target-hxq5c" (UID: "a8790322-f857-4238-9294-523d4cf3b5da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:00:55.291495 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:55.290605 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret\") pod \"global-pull-secret-syncer-n6c4h\" (UID: \"66af1c92-bb7a-4cda-bb2f-48fa2353b00e\") " pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:00:55.291495 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:55.290678 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs\") pod \"network-metrics-daemon-dq8k2\" (UID: \"f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a\") " pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:00:55.291495 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:55.290860 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:00:55.291495 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:55.290952 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs podName:f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a nodeName:}" failed. No retries permitted until 2026-07-07 15:00:57.290930938 +0000 UTC m=+6.107196976 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs") pod "network-metrics-daemon-dq8k2" (UID: "f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:00:55.291495 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:55.291397 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:00:55.291495 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:55.291454 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret podName:66af1c92-bb7a-4cda-bb2f-48fa2353b00e nodeName:}" failed. No retries permitted until 2026-07-07 15:00:57.291437492 +0000 UTC m=+6.107703517 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret") pod "global-pull-secret-syncer-n6c4h" (UID: "66af1c92-bb7a-4cda-bb2f-48fa2353b00e") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:00:55.608690 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:55.608603 2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 14:55:54 +0000 UTC" deadline="2028-01-04 10:50:30.258366868 +0000 UTC" Jul 07 15:00:55.608690 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:55.608643 2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13099h49m34.649728463s" Jul 07 15:00:55.785776 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:55.785690 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:00:55.785962 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:55.785881 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxq5c" podUID="a8790322-f857-4238-9294-523d4cf3b5da" Jul 07 15:00:55.786441 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:55.786397 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:00:55.786532 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:55.786501 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6c4h" podUID="66af1c92-bb7a-4cda-bb2f-48fa2353b00e" Jul 07 15:00:55.786602 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:55.786572 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:00:55.786822 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:55.786661 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dq8k2" podUID="f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a" Jul 07 15:00:55.812747 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:55.811952 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-39.ec2.internal" event={"ID":"e1813bc6b6359884c84e2d8d658d0cc0","Type":"ContainerStarted","Data":"c724c83fa1089b19916882a9a2092e3b09a999aa767c08aa895302bb06ecf80c"} Jul 07 15:00:55.816645 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:55.816602 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" event={"ID":"945a2dfd-7ceb-40d4-a093-1357ea0f74a1","Type":"ContainerStarted","Data":"67156fdd00360a24f34c78b6480edd0286386f715e4ff2f3749a9a7275585342"} Jul 07 15:00:55.822331 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:55.822301 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-39.ec2.internal" event={"ID":"1931978687adf475d28ebf31d0441510","Type":"ContainerStarted","Data":"da379e952ade5fae5238dc3cce29d203c0126e70ac04a6b65e2099a349d8862d"} Jul 07 15:00:55.845543 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:55.845478 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-39.ec2.internal" podStartSLOduration=2.84545525 podStartE2EDuration="2.84545525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:00:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:00:55.844614014 +0000 UTC m=+4.660880059" watchObservedRunningTime="2026-07-07 15:00:55.84545525 +0000 UTC m=+4.661721292" Jul 07 15:00:56.839259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:56.839206 2556 generic.go:356] "Generic (PLEG): container finished" podID="e1813bc6b6359884c84e2d8d658d0cc0" containerID="c724c83fa1089b19916882a9a2092e3b09a999aa767c08aa895302bb06ecf80c" exitCode=0 Jul 07 15:00:56.839854 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:56.839322 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-39.ec2.internal" event={"ID":"e1813bc6b6359884c84e2d8d658d0cc0","Type":"ContainerDied","Data":"c724c83fa1089b19916882a9a2092e3b09a999aa767c08aa895302bb06ecf80c"} Jul 07 15:00:56.879963 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:56.879889 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tzhn6" event={"ID":"ce32c7aa-dbf8-4ba2-bdd8-cfe8f4b6ca91","Type":"ContainerStarted","Data":"a4124a9575a8489a0acf23627dd97d7e5cbb8e3629d04b5d2bb017a9e5a5bf4e"} Jul 07 15:00:56.902968 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:56.895405 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tzhn6" podStartSLOduration=2.5463185839999998 podStartE2EDuration="4.895381895s" totalImagesPullingTime="2.349063311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:00:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:00:56.895250103 +0000 UTC m=+5.711516147" watchObservedRunningTime="2026-07-07 15:00:56.895381895 +0000 UTC m=+5.711647939" Jul 07 15:00:57.207271 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:57.206513 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6g68\" (UniqueName: \"kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68\") pod \"network-check-target-hxq5c\" (UID: \"a8790322-f857-4238-9294-523d4cf3b5da\") " pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:00:57.207271 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:57.206766 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:00:57.207271 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:57.206792 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:00:57.207271 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:57.206807 2556 projected.go:196] Error preparing data for projected volume kube-api-access-l6g68 for pod openshift-network-diagnostics/network-check-target-hxq5c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:00:57.207271 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:57.206875 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68 podName:a8790322-f857-4238-9294-523d4cf3b5da nodeName:}" failed. No retries permitted until 2026-07-07 15:01:01.206854749 +0000 UTC m=+10.023120778 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-l6g68" (UniqueName: "kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68") pod "network-check-target-hxq5c" (UID: "a8790322-f857-4238-9294-523d4cf3b5da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:00:57.308822 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:57.307992 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret\") pod \"global-pull-secret-syncer-n6c4h\" (UID: \"66af1c92-bb7a-4cda-bb2f-48fa2353b00e\") " pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:00:57.308822 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:57.308066 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs\") pod \"network-metrics-daemon-dq8k2\" (UID: \"f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a\") " pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:00:57.308822 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:57.308215 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:00:57.308822 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:57.308286 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs podName:f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a nodeName:}" failed. No retries permitted until 2026-07-07 15:01:01.308263839 +0000 UTC m=+10.124529879 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs") pod "network-metrics-daemon-dq8k2" (UID: "f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:00:57.308822 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:57.308713 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:00:57.308822 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:57.308770 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret podName:66af1c92-bb7a-4cda-bb2f-48fa2353b00e nodeName:}" failed. No retries permitted until 2026-07-07 15:01:01.308751671 +0000 UTC m=+10.125017690 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret") pod "global-pull-secret-syncer-n6c4h" (UID: "66af1c92-bb7a-4cda-bb2f-48fa2353b00e") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:00:57.785072 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:57.785038 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:00:57.785258 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:57.785175 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6c4h" podUID="66af1c92-bb7a-4cda-bb2f-48fa2353b00e" Jul 07 15:00:57.785622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:57.785602 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:00:57.785760 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:57.785724 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dq8k2" podUID="f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a" Jul 07 15:00:57.785844 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:57.785787 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:00:57.785901 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:57.785858 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxq5c" podUID="a8790322-f857-4238-9294-523d4cf3b5da" Jul 07 15:00:57.889256 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:57.889150 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-39.ec2.internal" event={"ID":"e1813bc6b6359884c84e2d8d658d0cc0","Type":"ContainerStarted","Data":"b03bb5f16bcf69f81864fd700147589c524b97265f7da8c00e2afa4dff1fac2e"} Jul 07 15:00:59.066211 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:59.065435 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tzhn6" Jul 07 15:00:59.067139 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:59.066898 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tzhn6" Jul 07 15:00:59.089438 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:59.089221 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-39.ec2.internal" podStartSLOduration=6.089199322 podStartE2EDuration="6.089199322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:00:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:00:57.905308426 +0000 UTC m=+6.721574470" watchObservedRunningTime="2026-07-07 15:00:59.089199322 +0000 UTC m=+7.905465366" Jul 07 15:00:59.782963 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:59.782251 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:00:59.782963 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:59.782304 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:00:59.782963 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:59.782415 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6c4h" podUID="66af1c92-bb7a-4cda-bb2f-48fa2353b00e" Jul 07 15:00:59.782963 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:59.782510 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dq8k2" podUID="f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a" Jul 07 15:00:59.782963 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:00:59.782595 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:00:59.782963 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:00:59.782675 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxq5c" podUID="a8790322-f857-4238-9294-523d4cf3b5da" Jul 07 15:01:01.243931 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:01.243802 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6g68\" (UniqueName: \"kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68\") pod \"network-check-target-hxq5c\" (UID: \"a8790322-f857-4238-9294-523d4cf3b5da\") " pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:01:01.244413 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:01.244002 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:01:01.244413 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:01.244022 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:01:01.244413 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:01.244036 2556 projected.go:196] Error preparing data for projected volume kube-api-access-l6g68 for pod openshift-network-diagnostics/network-check-target-hxq5c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:01:01.244413 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:01.244102 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68 podName:a8790322-f857-4238-9294-523d4cf3b5da nodeName:}" failed. No retries permitted until 2026-07-07 15:01:09.244080453 +0000 UTC m=+18.060346475 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-l6g68" (UniqueName: "kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68") pod "network-check-target-hxq5c" (UID: "a8790322-f857-4238-9294-523d4cf3b5da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:01:01.345754 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:01.345122 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret\") pod \"global-pull-secret-syncer-n6c4h\" (UID: \"66af1c92-bb7a-4cda-bb2f-48fa2353b00e\") " pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:01:01.345754 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:01.345181 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs\") pod \"network-metrics-daemon-dq8k2\" (UID: \"f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a\") " pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:01:01.345754 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:01.345286 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:01:01.345754 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:01.345339 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs podName:f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a nodeName:}" failed. No retries permitted until 2026-07-07 15:01:09.345319507 +0000 UTC m=+18.161585542 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs") pod "network-metrics-daemon-dq8k2" (UID: "f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:01:01.345754 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:01.345679 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:01:01.345754 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:01.345726 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret podName:66af1c92-bb7a-4cda-bb2f-48fa2353b00e nodeName:}" failed. No retries permitted until 2026-07-07 15:01:09.345714417 +0000 UTC m=+18.161980436 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret") pod "global-pull-secret-syncer-n6c4h" (UID: "66af1c92-bb7a-4cda-bb2f-48fa2353b00e") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:01:01.782661 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:01.782616 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:01:01.782872 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:01.782770 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6c4h" podUID="66af1c92-bb7a-4cda-bb2f-48fa2353b00e" Jul 07 15:01:01.784414 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:01.784386 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:01:01.784536 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:01.784501 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dq8k2" podUID="f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a" Jul 07 15:01:01.784701 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:01.784580 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:01:01.784701 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:01.784668 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxq5c" podUID="a8790322-f857-4238-9294-523d4cf3b5da" Jul 07 15:01:03.782798 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:03.782693 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:01:03.785933 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:03.784347 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:01:03.785933 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:03.784525 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxq5c" podUID="a8790322-f857-4238-9294-523d4cf3b5da" Jul 07 15:01:03.785933 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:03.784680 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6c4h" podUID="66af1c92-bb7a-4cda-bb2f-48fa2353b00e" Jul 07 15:01:03.785933 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:03.784903 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:01:03.787815 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:03.786322 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dq8k2" podUID="f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a" Jul 07 15:01:05.782171 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:05.782091 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:01:05.782171 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:05.782119 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:01:05.782608 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:05.782225 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxq5c" podUID="a8790322-f857-4238-9294-523d4cf3b5da" Jul 07 15:01:05.782608 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:05.782309 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:01:05.782608 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:05.782310 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6c4h" podUID="66af1c92-bb7a-4cda-bb2f-48fa2353b00e" Jul 07 15:01:05.782608 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:05.782396 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dq8k2" podUID="f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a" Jul 07 15:01:07.782411 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:07.782351 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:01:07.782909 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:07.782379 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:01:07.782909 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:07.782351 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:01:07.782909 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:07.782496 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dq8k2" podUID="f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a" Jul 07 15:01:07.782909 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:07.782624 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6c4h" podUID="66af1c92-bb7a-4cda-bb2f-48fa2353b00e" Jul 07 15:01:07.782909 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:07.782715 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxq5c" podUID="a8790322-f857-4238-9294-523d4cf3b5da" Jul 07 15:01:09.307915 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:09.307857 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6g68\" (UniqueName: \"kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68\") pod \"network-check-target-hxq5c\" (UID: \"a8790322-f857-4238-9294-523d4cf3b5da\") " pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:01:09.308409 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:09.308001 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:01:09.308409 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:09.308026 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:01:09.308409 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:09.308039 2556 projected.go:196] Error preparing data for projected volume kube-api-access-l6g68 for pod openshift-network-diagnostics/network-check-target-hxq5c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:01:09.308409 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:09.308105 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68 podName:a8790322-f857-4238-9294-523d4cf3b5da nodeName:}" failed. No retries permitted until 2026-07-07 15:01:25.30808471 +0000 UTC m=+34.124350750 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-l6g68" (UniqueName: "kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68") pod "network-check-target-hxq5c" (UID: "a8790322-f857-4238-9294-523d4cf3b5da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:01:09.408782 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:09.408746 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs\") pod \"network-metrics-daemon-dq8k2\" (UID: \"f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a\") " pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:01:09.408956 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:09.408808 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret\") pod \"global-pull-secret-syncer-n6c4h\" (UID: \"66af1c92-bb7a-4cda-bb2f-48fa2353b00e\") " pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:01:09.408956 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:09.408912 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:01:09.408956 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:09.408920 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:01:09.409093 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:09.408971 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs podName:f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a nodeName:}" failed. No retries permitted until 2026-07-07 15:01:25.408955747 +0000 UTC m=+34.225221764 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs") pod "network-metrics-daemon-dq8k2" (UID: "f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:01:09.409093 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:09.408985 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret podName:66af1c92-bb7a-4cda-bb2f-48fa2353b00e nodeName:}" failed. No retries permitted until 2026-07-07 15:01:25.408979057 +0000 UTC m=+34.225245074 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret") pod "global-pull-secret-syncer-n6c4h" (UID: "66af1c92-bb7a-4cda-bb2f-48fa2353b00e") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:01:09.782092 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:09.782018 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:01:09.782245 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:09.782018 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:01:09.782245 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:09.782146 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxq5c" podUID="a8790322-f857-4238-9294-523d4cf3b5da" Jul 07 15:01:09.782245 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:09.782029 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:01:09.782379 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:09.782240 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6c4h" podUID="66af1c92-bb7a-4cda-bb2f-48fa2353b00e" Jul 07 15:01:09.782379 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:09.782305 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dq8k2" podUID="f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a" Jul 07 15:01:10.834353 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:10.834296 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tzhn6" Jul 07 15:01:10.834810 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:10.834478 2556 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 15:01:10.834949 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:10.834930 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tzhn6" Jul 07 15:01:11.783277 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:11.783238 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:01:11.783466 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:11.783286 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:01:11.783466 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:11.783363 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6c4h" podUID="66af1c92-bb7a-4cda-bb2f-48fa2353b00e" Jul 07 15:01:11.783466 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:11.783454 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:01:11.783636 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:11.783453 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxq5c" podUID="a8790322-f857-4238-9294-523d4cf3b5da" Jul 07 15:01:11.783636 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:11.783547 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dq8k2" podUID="f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a" Jul 07 15:01:13.782586 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:13.782554 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:01:13.782978 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:13.782555 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:01:13.782978 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:13.782654 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxq5c" podUID="a8790322-f857-4238-9294-523d4cf3b5da" Jul 07 15:01:13.782978 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:13.782560 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:01:13.782978 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:13.782783 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dq8k2" podUID="f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a" Jul 07 15:01:13.782978 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:13.782849 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6c4h" podUID="66af1c92-bb7a-4cda-bb2f-48fa2353b00e" Jul 07 15:01:14.926580 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:14.926362 2556 generic.go:356] "Generic (PLEG): container finished" podID="0a6a7a17-0740-428b-b94c-e073a570c9a7" containerID="f3b279f7ba6bb49f58eceb9d3f66aceff90e0d26e7c44f9a8db853ca39dae729" exitCode=0 Jul 07 15:01:14.927493 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:14.926445 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lzp4x" event={"ID":"0a6a7a17-0740-428b-b94c-e073a570c9a7","Type":"ContainerDied","Data":"f3b279f7ba6bb49f58eceb9d3f66aceff90e0d26e7c44f9a8db853ca39dae729"} Jul 07 15:01:14.928399 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:14.928372 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-spm2h" event={"ID":"31976396-202a-47d9-b31f-9bf2efdfd835","Type":"ContainerStarted","Data":"94b9ac0a50c7ec4a398a0ac0af83ce28221c54ae67626544571146851c0c7865"} Jul 07 15:01:14.930174 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:14.930148 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" event={"ID":"c2a62882-861c-46c8-bfd4-d10ffd339dd1","Type":"ContainerStarted","Data":"465517e42e35b319f9aad1d6f49277087d980f52a7ce151cc03f48493b3c12d7"} Jul 07 15:01:14.933236 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:14.933209 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" event={"ID":"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5","Type":"ContainerStarted","Data":"9af9dc64379d49947e0d24511984c53105caa09548dcf98e9d13d0157a30782b"} Jul 07 15:01:14.933327 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:14.933274 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" event={"ID":"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5","Type":"ContainerStarted","Data":"67f5b5375c2786be5b44fed4a55f1cdb600987185fd46d2602b248ded2787d07"} Jul 07 15:01:14.933327 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:14.933308 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" event={"ID":"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5","Type":"ContainerStarted","Data":"453055e3b5cfd2dd3e02d00d6f980b52ea072c0d53427330cb55fcc967f9ca08"} Jul 07 15:01:14.933399 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:14.933335 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" event={"ID":"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5","Type":"ContainerStarted","Data":"ba10e14bddcaebece11dc99ffd47949ddb6bd1cb694f4866603f06346d022cd0"} Jul 07 15:01:14.933399 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:14.933369 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" event={"ID":"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5","Type":"ContainerStarted","Data":"3b42d87edf823829c3f8e50f7cd64114c638f20c14f6730540b3f7d7c17d3010"} Jul 07 15:01:14.935444 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:14.935417 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" event={"ID":"945a2dfd-7ceb-40d4-a093-1357ea0f74a1","Type":"ContainerStarted","Data":"bd3bc4bc1120e648a562fc40f7ce9aa04a398f540a70966b68c2f49e50c28795"} Jul 07 15:01:14.937041 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:14.937016 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-w6w74" event={"ID":"da182a28-bbef-4bfd-9273-b94f43794676","Type":"ContainerStarted","Data":"fca34542906e30d6d118592b368cbaa04820d3dc67f0eb3d203fdb49b6cacf59"} Jul 07 15:01:14.938483 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:14.938460 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vxlhb" event={"ID":"8466910e-c944-4c88-914b-596d470f31a6","Type":"ContainerStarted","Data":"e61304f34af82980d512341bbc9d3b11c7a39085714f109a543725d7f0b0c8a3"} Jul 07 15:01:14.965489 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:14.965443 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vxlhb" podStartSLOduration=2.963446293 podStartE2EDuration="22.965424261s" totalImagesPullingTime="20.001977968s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:00:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:14.964926447 +0000 UTC m=+23.781192486" watchObservedRunningTime="2026-07-07 15:01:14.965424261 +0000 UTC m=+23.781690303" Jul 07 15:01:14.980891 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:14.980338 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zs6ml" podStartSLOduration=2.985086239 podStartE2EDuration="22.980318307s" totalImagesPullingTime="19.995232068s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:00:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:14.980192705 +0000 UTC m=+23.796458745" watchObservedRunningTime="2026-07-07 15:01:14.980318307 +0000 UTC m=+23.796584348" Jul 07 15:01:14.993780 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:14.993741 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-w6w74" podStartSLOduration=3.0359675839999998 podStartE2EDuration="22.993727672s" totalImagesPullingTime="19.957760088s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:00:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:14.993569636 +0000 UTC m=+23.809835676" watchObservedRunningTime="2026-07-07 15:01:14.993727672 +0000 UTC m=+23.809993711" Jul 07 15:01:15.007972 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:15.007929 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-spm2h" podStartSLOduration=2.960257202 podStartE2EDuration="23.007912241s" totalImagesPullingTime="20.047655039s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:00:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:15.007682932 +0000 UTC m=+23.823948972" watchObservedRunningTime="2026-07-07 15:01:15.007912241 +0000 UTC m=+23.824178273" Jul 07 15:01:15.445696 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:15.445665 2556 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 15:01:15.660060 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:15.659903 2556 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T15:01:15.445684479Z","UUID":"34b246d0-c9c5-497f-beb7-5891c12bbae3","Handler":null,"Name":"","Endpoint":""} Jul 07 15:01:15.661366 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:15.661341 2556 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 15:01:15.661497 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:15.661380 2556 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 15:01:15.782180 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:15.782152 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:01:15.782383 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:15.782178 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:01:15.782383 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:15.782152 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:01:15.782383 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:15.782270 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxq5c" podUID="a8790322-f857-4238-9294-523d4cf3b5da" Jul 07 15:01:15.782544 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:15.782396 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dq8k2" podUID="f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a" Jul 07 15:01:15.782544 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:15.782487 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6c4h" podUID="66af1c92-bb7a-4cda-bb2f-48fa2353b00e" Jul 07 15:01:15.943376 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:15.943279 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" event={"ID":"945a2dfd-7ceb-40d4-a093-1357ea0f74a1","Type":"ContainerStarted","Data":"dd31f17a19df4a0466bc40ad217e21a2ed9bd6c0c625dd865261ad930d168b43"} Jul 07 15:01:15.945069 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:15.945036 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-47cnz" event={"ID":"c17c3a38-de46-4eae-888f-c4424a8e139e","Type":"ContainerStarted","Data":"df17722e886ab5992f3bad3eec1a944c27cb56e2b184826dd9521927ebf41a5c"} Jul 07 15:01:15.948482 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:15.948442 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" event={"ID":"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5","Type":"ContainerStarted","Data":"f3f37c9d370f30889324d472f9582a869c0310d6ce9e5a43274cc58b3ab950d2"} Jul 07 15:01:15.963410 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:15.963360 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-47cnz" podStartSLOduration=3.958316722 podStartE2EDuration="23.96334223s" totalImagesPullingTime="20.005025508s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:00:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:15.962702825 +0000 UTC m=+24.778968865" watchObservedRunningTime="2026-07-07 15:01:15.96334223 +0000 UTC m=+24.779608269" Jul 07 15:01:17.782907 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:17.782711 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:01:17.783323 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:17.782720 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:01:17.783323 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:17.782979 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6c4h" podUID="66af1c92-bb7a-4cda-bb2f-48fa2353b00e" Jul 07 15:01:17.783323 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:17.782766 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:01:17.783323 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:17.783055 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dq8k2" podUID="f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a" Jul 07 15:01:17.783323 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:17.783115 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxq5c" podUID="a8790322-f857-4238-9294-523d4cf3b5da" Jul 07 15:01:17.957005 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:17.956968 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" event={"ID":"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5","Type":"ContainerStarted","Data":"740e7c9a56703787cee39121345e59551215470b861e10ff6718d29590074af5"} Jul 07 15:01:17.959208 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:17.959181 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" event={"ID":"945a2dfd-7ceb-40d4-a093-1357ea0f74a1","Type":"ContainerStarted","Data":"852a6c966848a199c010148c87fb7ee72d18d719c9a09acb6ee5c7f3137262da"} Jul 07 15:01:17.978449 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:17.978393 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svcgm" podStartSLOduration=4.254794154 podStartE2EDuration="25.97837824s" totalImagesPullingTime="21.723584086s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:00:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:17.978359504 +0000 UTC m=+26.794625545" watchObservedRunningTime="2026-07-07 15:01:17.97837824 +0000 UTC m=+26.794644270" Jul 07 15:01:19.782364 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:19.782284 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:01:19.782364 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:19.782328 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:01:19.783095 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:19.782284 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:01:19.783095 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:19.782419 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6c4h" podUID="66af1c92-bb7a-4cda-bb2f-48fa2353b00e" Jul 07 15:01:19.783095 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:19.782466 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxq5c" podUID="a8790322-f857-4238-9294-523d4cf3b5da" Jul 07 15:01:19.783095 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:19.782558 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dq8k2" podUID="f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a" Jul 07 15:01:20.968720 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:20.968515 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lzp4x" event={"ID":"0a6a7a17-0740-428b-b94c-e073a570c9a7","Type":"ContainerStarted","Data":"b0460be08cff2d3274a872b40d894434491cb1013d602cdf683ed91e16465acf"} Jul 07 15:01:20.972482 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:20.972454 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" event={"ID":"3e8f59d5-d4a6-4685-a7f3-ff556ff165a5","Type":"ContainerStarted","Data":"65accffb1554c69f58a5cc8565acee1a532720648e920f7e3809bb3520dc0e52"} Jul 07 15:01:20.972733 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:20.972710 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:01:20.972891 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:20.972779 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:01:20.987740 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:20.987716 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:01:21.017861 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:21.017814 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" podStartSLOduration=8.629640534 podStartE2EDuration="29.017796414s" totalImagesPullingTime="20.38815588s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:00:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:21.017383076 +0000 UTC m=+29.833649114" watchObservedRunningTime="2026-07-07 15:01:21.017796414 +0000 UTC m=+29.834062455" Jul 07 15:01:21.784032 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:21.783509 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:01:21.784032 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:21.783554 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:01:21.784032 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:21.783618 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6c4h" podUID="66af1c92-bb7a-4cda-bb2f-48fa2353b00e" Jul 07 15:01:21.784032 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:21.783689 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:01:21.784032 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:21.783786 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxq5c" podUID="a8790322-f857-4238-9294-523d4cf3b5da" Jul 07 15:01:21.784032 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:21.783894 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dq8k2" podUID="f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a" Jul 07 15:01:21.979854 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:21.979811 2556 generic.go:356] "Generic (PLEG): container finished" podID="0a6a7a17-0740-428b-b94c-e073a570c9a7" containerID="b0460be08cff2d3274a872b40d894434491cb1013d602cdf683ed91e16465acf" exitCode=0 Jul 07 15:01:21.980351 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:21.979903 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lzp4x" event={"ID":"0a6a7a17-0740-428b-b94c-e073a570c9a7","Type":"ContainerDied","Data":"b0460be08cff2d3274a872b40d894434491cb1013d602cdf683ed91e16465acf"} Jul 07 15:01:21.980492 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:21.980457 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:01:21.997252 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:21.997226 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:01:22.527913 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:22.527862 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n6c4h"] Jul 07 15:01:22.528088 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:22.528054 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:01:22.528210 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:22.528187 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6c4h" podUID="66af1c92-bb7a-4cda-bb2f-48fa2353b00e" Jul 07 15:01:22.528506 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:22.528484 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hxq5c"] Jul 07 15:01:22.528594 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:22.528586 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:01:22.528704 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:22.528685 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxq5c" podUID="a8790322-f857-4238-9294-523d4cf3b5da" Jul 07 15:01:22.530396 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:22.530375 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dq8k2"] Jul 07 15:01:22.530471 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:22.530453 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:01:22.530537 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:22.530521 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dq8k2" podUID="f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a" Jul 07 15:01:23.783014 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:23.782838 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:01:23.783653 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:23.783091 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxq5c" podUID="a8790322-f857-4238-9294-523d4cf3b5da" Jul 07 15:01:23.985559 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:23.985517 2556 generic.go:356] "Generic (PLEG): container finished" podID="0a6a7a17-0740-428b-b94c-e073a570c9a7" containerID="478abe90a7c869efb6120a00067b43769bf5f6677f773feaf2298ae9c613035d" exitCode=0 Jul 07 15:01:23.985559 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:23.985555 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lzp4x" event={"ID":"0a6a7a17-0740-428b-b94c-e073a570c9a7","Type":"ContainerDied","Data":"478abe90a7c869efb6120a00067b43769bf5f6677f773feaf2298ae9c613035d"} Jul 07 15:01:24.782879 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:24.782839 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:01:24.783045 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:24.782988 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dq8k2" podUID="f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a" Jul 07 15:01:24.783110 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:24.783047 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:01:24.783570 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:24.783155 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6c4h" podUID="66af1c92-bb7a-4cda-bb2f-48fa2353b00e" Jul 07 15:01:25.323188 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:25.323094 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6g68\" (UniqueName: \"kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68\") pod \"network-check-target-hxq5c\" (UID: \"a8790322-f857-4238-9294-523d4cf3b5da\") " pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:01:25.323323 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:25.323273 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 15:01:25.323323 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:25.323299 2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 15:01:25.323323 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:25.323309 2556 projected.go:196] Error preparing data for projected volume kube-api-access-l6g68 for pod openshift-network-diagnostics/network-check-target-hxq5c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:01:25.323429 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:25.323362 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68 podName:a8790322-f857-4238-9294-523d4cf3b5da nodeName:}" failed. No retries permitted until 2026-07-07 15:01:57.32334723 +0000 UTC m=+66.139613248 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-l6g68" (UniqueName: "kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68") pod "network-check-target-hxq5c" (UID: "a8790322-f857-4238-9294-523d4cf3b5da") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 15:01:25.423925 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:25.423887 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs\") pod \"network-metrics-daemon-dq8k2\" (UID: \"f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a\") " pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:01:25.424093 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:25.423961 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret\") pod \"global-pull-secret-syncer-n6c4h\" (UID: \"66af1c92-bb7a-4cda-bb2f-48fa2353b00e\") " pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:01:25.424093 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:25.424048 2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 15:01:25.424177 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:25.424104 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret podName:66af1c92-bb7a-4cda-bb2f-48fa2353b00e nodeName:}" failed. No retries permitted until 2026-07-07 15:01:57.424091443 +0000 UTC m=+66.240357460 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret") pod "global-pull-secret-syncer-n6c4h" (UID: "66af1c92-bb7a-4cda-bb2f-48fa2353b00e") : object "kube-system"/"original-pull-secret" not registered Jul 07 15:01:25.424177 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:25.424048 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:01:25.424177 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:25.424175 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs podName:f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a nodeName:}" failed. No retries permitted until 2026-07-07 15:01:57.424161056 +0000 UTC m=+66.240427090 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs") pod "network-metrics-daemon-dq8k2" (UID: "f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 15:01:25.781996 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:25.781961 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:01:25.782167 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:25.782065 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hxq5c" podUID="a8790322-f857-4238-9294-523d4cf3b5da" Jul 07 15:01:25.991920 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:25.991883 2556 generic.go:356] "Generic (PLEG): container finished" podID="0a6a7a17-0740-428b-b94c-e073a570c9a7" containerID="5105d3b6638e05c02ddad315d59ef82d1768a06c499f1140fae637a568b0110f" exitCode=0 Jul 07 15:01:25.992289 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:25.991930 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lzp4x" event={"ID":"0a6a7a17-0740-428b-b94c-e073a570c9a7","Type":"ContainerDied","Data":"5105d3b6638e05c02ddad315d59ef82d1768a06c499f1140fae637a568b0110f"} Jul 07 15:01:26.782026 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:26.781996 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:01:26.782194 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:26.782098 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dq8k2" podUID="f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a" Jul 07 15:01:26.782194 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:26.782156 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:01:26.782266 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:26.782247 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n6c4h" podUID="66af1c92-bb7a-4cda-bb2f-48fa2353b00e" Jul 07 15:01:26.967428 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:26.967346 2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-39.ec2.internal" event="NodeReady" Jul 07 15:01:26.967575 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:26.967452 2556 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 15:01:27.010110 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.010078 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-l7998"] Jul 07 15:01:27.012282 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.012260 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l7998" Jul 07 15:01:27.016221 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.015527 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bp52v\"" Jul 07 15:01:27.016221 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.015666 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-cm8wk"] Jul 07 15:01:27.016221 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.016058 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 15:01:27.016777 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.016754 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 15:01:27.018088 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.018067 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cm8wk" Jul 07 15:01:27.020339 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.020319 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 15:01:27.020967 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.020946 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-j8pj6\"" Jul 07 15:01:27.021058 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.020999 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 15:01:27.021058 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.021030 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 15:01:27.022752 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.022731 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l7998"] Jul 07 15:01:27.033316 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.033291 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cm8wk"] Jul 07 15:01:27.138248 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.138218 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b040cb56-952e-4172-a484-89ba96e76a45-cert\") pod \"ingress-canary-cm8wk\" (UID: \"b040cb56-952e-4172-a484-89ba96e76a45\") " pod="openshift-ingress-canary/ingress-canary-cm8wk" Jul 07 15:01:27.138422 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.138284 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls\") pod \"dns-default-l7998\" (UID: \"1266e9ff-4ff3-4177-8402-1550a338121a\") " pod="openshift-dns/dns-default-l7998" Jul 07 15:01:27.138422 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.138344 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1266e9ff-4ff3-4177-8402-1550a338121a-tmp-dir\") pod \"dns-default-l7998\" (UID: \"1266e9ff-4ff3-4177-8402-1550a338121a\") " pod="openshift-dns/dns-default-l7998" Jul 07 15:01:27.138523 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.138458 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzm7s\" (UniqueName: \"kubernetes.io/projected/b040cb56-952e-4172-a484-89ba96e76a45-kube-api-access-qzm7s\") pod \"ingress-canary-cm8wk\" (UID: \"b040cb56-952e-4172-a484-89ba96e76a45\") " pod="openshift-ingress-canary/ingress-canary-cm8wk" Jul 07 15:01:27.138523 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.138505 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1266e9ff-4ff3-4177-8402-1550a338121a-config-volume\") pod \"dns-default-l7998\" (UID: \"1266e9ff-4ff3-4177-8402-1550a338121a\") " pod="openshift-dns/dns-default-l7998" Jul 07 15:01:27.138611 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.138529 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2lwr\" (UniqueName: \"kubernetes.io/projected/1266e9ff-4ff3-4177-8402-1550a338121a-kube-api-access-d2lwr\") pod \"dns-default-l7998\" (UID: \"1266e9ff-4ff3-4177-8402-1550a338121a\") " pod="openshift-dns/dns-default-l7998" Jul 07 15:01:27.239635 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.239539 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls\") pod \"dns-default-l7998\" (UID: \"1266e9ff-4ff3-4177-8402-1550a338121a\") " pod="openshift-dns/dns-default-l7998" Jul 07 15:01:27.239635 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.239586 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1266e9ff-4ff3-4177-8402-1550a338121a-tmp-dir\") pod \"dns-default-l7998\" (UID: \"1266e9ff-4ff3-4177-8402-1550a338121a\") " pod="openshift-dns/dns-default-l7998" Jul 07 15:01:27.239908 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.239645 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzm7s\" (UniqueName: \"kubernetes.io/projected/b040cb56-952e-4172-a484-89ba96e76a45-kube-api-access-qzm7s\") pod \"ingress-canary-cm8wk\" (UID: \"b040cb56-952e-4172-a484-89ba96e76a45\") " pod="openshift-ingress-canary/ingress-canary-cm8wk" Jul 07 15:01:27.239908 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.239693 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1266e9ff-4ff3-4177-8402-1550a338121a-config-volume\") pod \"dns-default-l7998\" (UID: \"1266e9ff-4ff3-4177-8402-1550a338121a\") " pod="openshift-dns/dns-default-l7998" Jul 07 15:01:27.239908 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.239718 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2lwr\" (UniqueName: \"kubernetes.io/projected/1266e9ff-4ff3-4177-8402-1550a338121a-kube-api-access-d2lwr\") pod \"dns-default-l7998\" (UID: \"1266e9ff-4ff3-4177-8402-1550a338121a\") " pod="openshift-dns/dns-default-l7998" Jul 07 15:01:27.239908 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:27.239726 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:01:27.239908 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.239776 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b040cb56-952e-4172-a484-89ba96e76a45-cert\") pod \"ingress-canary-cm8wk\" (UID: \"b040cb56-952e-4172-a484-89ba96e76a45\") " pod="openshift-ingress-canary/ingress-canary-cm8wk" Jul 07 15:01:27.239908 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:27.239810 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls podName:1266e9ff-4ff3-4177-8402-1550a338121a nodeName:}" failed. No retries permitted until 2026-07-07 15:01:27.739788288 +0000 UTC m=+36.556054328 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls") pod "dns-default-l7998" (UID: "1266e9ff-4ff3-4177-8402-1550a338121a") : secret "dns-default-metrics-tls" not found Jul 07 15:01:27.240190 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.240098 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1266e9ff-4ff3-4177-8402-1550a338121a-tmp-dir\") pod \"dns-default-l7998\" (UID: \"1266e9ff-4ff3-4177-8402-1550a338121a\") " pod="openshift-dns/dns-default-l7998" Jul 07 15:01:27.240404 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.240382 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1266e9ff-4ff3-4177-8402-1550a338121a-config-volume\") pod \"dns-default-l7998\" (UID: \"1266e9ff-4ff3-4177-8402-1550a338121a\") " pod="openshift-dns/dns-default-l7998" Jul 07 15:01:27.244694 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.244671 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b040cb56-952e-4172-a484-89ba96e76a45-cert\") pod \"ingress-canary-cm8wk\" (UID: \"b040cb56-952e-4172-a484-89ba96e76a45\") " pod="openshift-ingress-canary/ingress-canary-cm8wk" Jul 07 15:01:27.249053 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.249029 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2lwr\" (UniqueName: \"kubernetes.io/projected/1266e9ff-4ff3-4177-8402-1550a338121a-kube-api-access-d2lwr\") pod \"dns-default-l7998\" (UID: \"1266e9ff-4ff3-4177-8402-1550a338121a\") " pod="openshift-dns/dns-default-l7998" Jul 07 15:01:27.249772 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.249739 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzm7s\" (UniqueName: \"kubernetes.io/projected/b040cb56-952e-4172-a484-89ba96e76a45-kube-api-access-qzm7s\") pod \"ingress-canary-cm8wk\" (UID: \"b040cb56-952e-4172-a484-89ba96e76a45\") " pod="openshift-ingress-canary/ingress-canary-cm8wk" Jul 07 15:01:27.330381 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.330347 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cm8wk" Jul 07 15:01:27.460843 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.460670 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cm8wk"] Jul 07 15:01:27.464257 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:01:27.464224 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb040cb56_952e_4172_a484_89ba96e76a45.slice/crio-de74ebaed2dae536310a794e3c4913142eb0b4734fca4ddbe8d5ec5cebea0ccb WatchSource:0}: Error finding container de74ebaed2dae536310a794e3c4913142eb0b4734fca4ddbe8d5ec5cebea0ccb: Status 404 returned error can't find the container with id de74ebaed2dae536310a794e3c4913142eb0b4734fca4ddbe8d5ec5cebea0ccb Jul 07 15:01:27.744049 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.744016 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls\") pod \"dns-default-l7998\" (UID: \"1266e9ff-4ff3-4177-8402-1550a338121a\") " pod="openshift-dns/dns-default-l7998" Jul 07 15:01:27.744253 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:27.744155 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:01:27.744253 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:27.744224 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls podName:1266e9ff-4ff3-4177-8402-1550a338121a nodeName:}" failed. No retries permitted until 2026-07-07 15:01:28.744206551 +0000 UTC m=+37.560472588 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls") pod "dns-default-l7998" (UID: "1266e9ff-4ff3-4177-8402-1550a338121a") : secret "dns-default-metrics-tls" not found Jul 07 15:01:27.782368 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.782334 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:01:27.784818 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.784779 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 15:01:27.785579 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.785559 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 15:01:27.785709 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.785621 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2msz5\"" Jul 07 15:01:27.997960 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:27.997856 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cm8wk" event={"ID":"b040cb56-952e-4172-a484-89ba96e76a45","Type":"ContainerStarted","Data":"de74ebaed2dae536310a794e3c4913142eb0b4734fca4ddbe8d5ec5cebea0ccb"} Jul 07 15:01:28.752369 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:28.752331 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls\") pod \"dns-default-l7998\" (UID: \"1266e9ff-4ff3-4177-8402-1550a338121a\") " pod="openshift-dns/dns-default-l7998" Jul 07 15:01:28.752883 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:28.752495 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:01:28.752883 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:28.752576 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls podName:1266e9ff-4ff3-4177-8402-1550a338121a nodeName:}" failed. No retries permitted until 2026-07-07 15:01:30.75255377 +0000 UTC m=+39.568819792 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls") pod "dns-default-l7998" (UID: "1266e9ff-4ff3-4177-8402-1550a338121a") : secret "dns-default-metrics-tls" not found Jul 07 15:01:28.782308 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:28.782274 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:01:28.782494 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:28.782321 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:01:28.786459 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:28.786437 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 15:01:28.786459 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:28.786458 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mbwr6\"" Jul 07 15:01:28.787099 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:28.787080 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 15:01:30.766357 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:30.766312 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls\") pod \"dns-default-l7998\" (UID: \"1266e9ff-4ff3-4177-8402-1550a338121a\") " pod="openshift-dns/dns-default-l7998" Jul 07 15:01:30.766826 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:30.766490 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:01:30.766826 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:30.766567 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls podName:1266e9ff-4ff3-4177-8402-1550a338121a nodeName:}" failed. No retries permitted until 2026-07-07 15:01:34.766547103 +0000 UTC m=+43.582813120 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls") pod "dns-default-l7998" (UID: "1266e9ff-4ff3-4177-8402-1550a338121a") : secret "dns-default-metrics-tls" not found Jul 07 15:01:31.005852 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:31.005811 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cm8wk" event={"ID":"b040cb56-952e-4172-a484-89ba96e76a45","Type":"ContainerStarted","Data":"896d4ea284822db51f40a3fbbe04230759e5b603cb2ffe49ee7130e135b3d12c"} Jul 07 15:01:31.021993 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:31.021899 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-cm8wk" podStartSLOduration=2.480977305 podStartE2EDuration="5.021883719s" totalImagesPullingTime="2.540906414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:31.021558969 +0000 UTC m=+39.837825008" watchObservedRunningTime="2026-07-07 15:01:31.021883719 +0000 UTC m=+39.838149748" Jul 07 15:01:34.015396 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:34.015360 2556 generic.go:356] "Generic (PLEG): container finished" podID="0a6a7a17-0740-428b-b94c-e073a570c9a7" containerID="8209bae78a0ca1c966ae5f0153a8375d3d57baf4373de243d233a41d1f0d82e8" exitCode=0 Jul 07 15:01:34.015781 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:34.015408 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lzp4x" event={"ID":"0a6a7a17-0740-428b-b94c-e073a570c9a7","Type":"ContainerDied","Data":"8209bae78a0ca1c966ae5f0153a8375d3d57baf4373de243d233a41d1f0d82e8"} Jul 07 15:01:34.796967 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:34.796928 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls\") pod \"dns-default-l7998\" (UID: \"1266e9ff-4ff3-4177-8402-1550a338121a\") " pod="openshift-dns/dns-default-l7998" Jul 07 15:01:34.797134 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:34.797082 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:01:34.797177 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:34.797147 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls podName:1266e9ff-4ff3-4177-8402-1550a338121a nodeName:}" failed. No retries permitted until 2026-07-07 15:01:42.79713153 +0000 UTC m=+51.613397547 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls") pod "dns-default-l7998" (UID: "1266e9ff-4ff3-4177-8402-1550a338121a") : secret "dns-default-metrics-tls" not found Jul 07 15:01:35.020861 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:35.020832 2556 generic.go:356] "Generic (PLEG): container finished" podID="0a6a7a17-0740-428b-b94c-e073a570c9a7" containerID="1c3564f089b0d4ff9dbcdf29329cd7d5f0c1e75db322764db5825e0f4b5c2f54" exitCode=0 Jul 07 15:01:35.021228 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:35.020912 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lzp4x" event={"ID":"0a6a7a17-0740-428b-b94c-e073a570c9a7","Type":"ContainerDied","Data":"1c3564f089b0d4ff9dbcdf29329cd7d5f0c1e75db322764db5825e0f4b5c2f54"} Jul 07 15:01:36.026788 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:36.026751 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lzp4x" event={"ID":"0a6a7a17-0740-428b-b94c-e073a570c9a7","Type":"ContainerStarted","Data":"c7633750baec0c1bbe05883d1a7fd1e21ce38936e81783faaf1afec60cc4e94b"} Jul 07 15:01:36.051224 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:36.051161 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lzp4x" podStartSLOduration=8.200160855 podStartE2EDuration="44.05114537s" totalImagesPullingTime="35.850984515s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:00:52 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:01:36.050717054 +0000 UTC m=+44.866983094" watchObservedRunningTime="2026-07-07 15:01:36.05114537 +0000 UTC m=+44.867411409" Jul 07 15:01:42.853457 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:42.853408 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls\") pod \"dns-default-l7998\" (UID: \"1266e9ff-4ff3-4177-8402-1550a338121a\") " pod="openshift-dns/dns-default-l7998" Jul 07 15:01:42.853960 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:42.853564 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:01:42.853960 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:42.853631 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls podName:1266e9ff-4ff3-4177-8402-1550a338121a nodeName:}" failed. No retries permitted until 2026-07-07 15:01:58.853611497 +0000 UTC m=+67.669877534 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls") pod "dns-default-l7998" (UID: "1266e9ff-4ff3-4177-8402-1550a338121a") : secret "dns-default-metrics-tls" not found Jul 07 15:01:53.996186 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:53.996152 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hqqpl" Jul 07 15:01:57.353403 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:57.353367 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6g68\" (UniqueName: \"kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68\") pod \"network-check-target-hxq5c\" (UID: \"a8790322-f857-4238-9294-523d4cf3b5da\") " pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:01:57.356040 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:57.356015 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 15:01:57.366454 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:57.366432 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 15:01:57.378102 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:57.378078 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6g68\" (UniqueName: \"kubernetes.io/projected/a8790322-f857-4238-9294-523d4cf3b5da-kube-api-access-l6g68\") pod \"network-check-target-hxq5c\" (UID: \"a8790322-f857-4238-9294-523d4cf3b5da\") " pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:01:57.454051 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:57.454013 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs\") pod \"network-metrics-daemon-dq8k2\" (UID: \"f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a\") " pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:01:57.454248 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:57.454081 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret\") pod \"global-pull-secret-syncer-n6c4h\" (UID: \"66af1c92-bb7a-4cda-bb2f-48fa2353b00e\") " pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:01:57.456641 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:57.456624 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 15:01:57.456641 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:57.456635 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 15:01:57.464638 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:57.464618 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 07 15:01:57.464703 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:57.464686 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs podName:f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a nodeName:}" failed. No retries permitted until 2026-07-07 15:03:01.464668499 +0000 UTC m=+130.280934516 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs") pod "network-metrics-daemon-dq8k2" (UID: "f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a") : secret "metrics-daemon-secret" not found Jul 07 15:01:57.466644 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:57.466626 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66af1c92-bb7a-4cda-bb2f-48fa2353b00e-original-pull-secret\") pod \"global-pull-secret-syncer-n6c4h\" (UID: \"66af1c92-bb7a-4cda-bb2f-48fa2353b00e\") " pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:01:57.496607 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:57.496578 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2msz5\"" Jul 07 15:01:57.504705 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:57.504684 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:01:57.599245 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:57.599220 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n6c4h" Jul 07 15:01:57.652195 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:57.652163 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hxq5c"] Jul 07 15:01:57.656587 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:01:57.656558 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8790322_f857_4238_9294_523d4cf3b5da.slice/crio-088bbb7990b359bceb193344afdbbdcab31469862f6a638ebd607ef94f8c81b8 WatchSource:0}: Error finding container 088bbb7990b359bceb193344afdbbdcab31469862f6a638ebd607ef94f8c81b8: Status 404 returned error can't find the container with id 088bbb7990b359bceb193344afdbbdcab31469862f6a638ebd607ef94f8c81b8 Jul 07 15:01:57.723664 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:57.723631 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n6c4h"] Jul 07 15:01:57.726819 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:01:57.726792 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66af1c92_bb7a_4cda_bb2f_48fa2353b00e.slice/crio-3385157c1a3351193b8fa87598f7f0e2708c32c8c4631c1dd6f0b173a7eea467 WatchSource:0}: Error finding container 3385157c1a3351193b8fa87598f7f0e2708c32c8c4631c1dd6f0b173a7eea467: Status 404 returned error can't find the container with id 3385157c1a3351193b8fa87598f7f0e2708c32c8c4631c1dd6f0b173a7eea467 Jul 07 15:01:58.078836 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:58.078796 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n6c4h" event={"ID":"66af1c92-bb7a-4cda-bb2f-48fa2353b00e","Type":"ContainerStarted","Data":"3385157c1a3351193b8fa87598f7f0e2708c32c8c4631c1dd6f0b173a7eea467"} Jul 07 15:01:58.079851 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:58.079821 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hxq5c" event={"ID":"a8790322-f857-4238-9294-523d4cf3b5da","Type":"ContainerStarted","Data":"088bbb7990b359bceb193344afdbbdcab31469862f6a638ebd607ef94f8c81b8"} Jul 07 15:01:58.862410 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:01:58.862370 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls\") pod \"dns-default-l7998\" (UID: \"1266e9ff-4ff3-4177-8402-1550a338121a\") " pod="openshift-dns/dns-default-l7998" Jul 07 15:01:58.862907 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:58.862522 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:01:58.862907 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:01:58.862608 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls podName:1266e9ff-4ff3-4177-8402-1550a338121a nodeName:}" failed. No retries permitted until 2026-07-07 15:02:30.862588228 +0000 UTC m=+99.678854245 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls") pod "dns-default-l7998" (UID: "1266e9ff-4ff3-4177-8402-1550a338121a") : secret "dns-default-metrics-tls" not found Jul 07 15:02:03.094817 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:02:03.094778 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n6c4h" event={"ID":"66af1c92-bb7a-4cda-bb2f-48fa2353b00e","Type":"ContainerStarted","Data":"50a256af10728c8c8aa82c3c4fb7a06335fe6c5c3fa61e3e3f497a407f76ae93"} Jul 07 15:02:03.096210 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:02:03.096185 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hxq5c" event={"ID":"a8790322-f857-4238-9294-523d4cf3b5da","Type":"ContainerStarted","Data":"3e793a235c9c74948931fc2f965e105a8e252174d193b995c2229293812d3e78"} Jul 07 15:02:03.096341 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:02:03.096234 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:02:03.110055 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:02:03.110007 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-n6c4h" podStartSLOduration=66.474366826 podStartE2EDuration="1m11.109994127s" totalImagesPullingTime="4.635627301s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:00:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:02:03.10949774 +0000 UTC m=+71.925763780" watchObservedRunningTime="2026-07-07 15:02:03.109994127 +0000 UTC m=+71.926260165" Jul 07 15:02:03.123498 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:02:03.123453 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hxq5c" podStartSLOduration=66.423242675 podStartE2EDuration="1m11.12343879s" totalImagesPullingTime="4.700196115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:00:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:02:03.122948043 +0000 UTC m=+71.939214086" watchObservedRunningTime="2026-07-07 15:02:03.12343879 +0000 UTC m=+71.939704808" Jul 07 15:02:30.882519 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:02:30.882457 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls\") pod \"dns-default-l7998\" (UID: \"1266e9ff-4ff3-4177-8402-1550a338121a\") " pod="openshift-dns/dns-default-l7998" Jul 07 15:02:30.883035 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:02:30.882591 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:02:30.883035 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:02:30.882652 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls podName:1266e9ff-4ff3-4177-8402-1550a338121a nodeName:}" failed. No retries permitted until 2026-07-07 15:03:34.882634304 +0000 UTC m=+163.698900335 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls") pod "dns-default-l7998" (UID: "1266e9ff-4ff3-4177-8402-1550a338121a") : secret "dns-default-metrics-tls" not found Jul 07 15:02:34.100400 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:02:34.100372 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hxq5c" Jul 07 15:03:01.487975 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:03:01.487923 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs\") pod \"network-metrics-daemon-dq8k2\" (UID: \"f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a\") " pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:03:01.488463 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:03:01.488060 2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 07 15:03:01.488463 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:03:01.488118 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs podName:f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a nodeName:}" failed. No retries permitted until 2026-07-07 15:05:03.488103726 +0000 UTC m=+252.304369743 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs") pod "network-metrics-daemon-dq8k2" (UID: "f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a") : secret "metrics-daemon-secret" not found Jul 07 15:03:30.025014 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:03:30.024964 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-l7998" podUID="1266e9ff-4ff3-4177-8402-1550a338121a" Jul 07 15:03:30.299930 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:03:30.299839 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l7998" Jul 07 15:03:31.793611 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:03:31.793577 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-dq8k2" podUID="f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a" Jul 07 15:03:34.908768 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:03:34.908724 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls\") pod \"dns-default-l7998\" (UID: \"1266e9ff-4ff3-4177-8402-1550a338121a\") " pod="openshift-dns/dns-default-l7998" Jul 07 15:03:34.909154 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:03:34.908909 2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 15:03:34.909154 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:03:34.908983 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls podName:1266e9ff-4ff3-4177-8402-1550a338121a nodeName:}" failed. No retries permitted until 2026-07-07 15:05:36.908966692 +0000 UTC m=+285.725232708 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls") pod "dns-default-l7998" (UID: "1266e9ff-4ff3-4177-8402-1550a338121a") : secret "dns-default-metrics-tls" not found Jul 07 15:03:44.781926 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:03:44.781810 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:04:42.973488 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:42.973453 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-7qqkl"] Jul 07 15:04:42.975360 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:42.975340 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" Jul 07 15:04:42.978600 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:42.978580 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 07 15:04:42.978910 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:42.978888 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 15:04:42.979001 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:42.978929 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 15:04:42.979001 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:42.978956 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 07 15:04:42.979549 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:42.979534 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-68qfl\"" Jul 07 15:04:42.983182 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:42.983154 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-z4h8q"] Jul 07 15:04:42.984937 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:42.984918 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5c76867bd5-72zmj"] Jul 07 15:04:42.985057 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:42.985011 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" Jul 07 15:04:42.986678 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:42.986660 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:43.005175 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.005154 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-7qqkl"] Jul 07 15:04:43.012710 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.012690 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 07 15:04:43.013161 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.013142 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 07 15:04:43.013254 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.013214 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:04:43.013254 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.013223 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 15:04:43.013254 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.013237 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 07 15:04:43.013510 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.013497 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 07 15:04:43.013719 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.013706 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 07 15:04:43.013764 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.013708 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 07 15:04:43.013942 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.013926 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 07 15:04:43.014031 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.014018 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4pm6j\"" Jul 07 15:04:43.014226 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.014211 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 15:04:43.014302 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.014258 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 07 15:04:43.014356 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.014258 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-wlj5j\"" Jul 07 15:04:43.032642 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.032618 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 07 15:04:43.032958 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.032939 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-z4h8q"] Jul 07 15:04:43.037648 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.037628 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5c76867bd5-72zmj"] Jul 07 15:04:43.051097 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.051075 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89ce2379-8dcb-4fde-8423-47dc9bf4512f-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-7qqkl\" (UID: \"89ce2379-8dcb-4fde-8423-47dc9bf4512f\") " pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" Jul 07 15:04:43.051221 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.051103 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-service-ca-bundle\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:43.051221 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.051120 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-metrics-certs\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:43.051221 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.051148 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4faf5ae7-9e9a-41ac-aad2-9808d11d4c42-config\") pod \"console-operator-68b757fb67-z4h8q\" (UID: \"4faf5ae7-9e9a-41ac-aad2-9808d11d4c42\") " pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" Jul 07 15:04:43.051221 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.051170 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4faf5ae7-9e9a-41ac-aad2-9808d11d4c42-trusted-ca\") pod \"console-operator-68b757fb67-z4h8q\" (UID: \"4faf5ae7-9e9a-41ac-aad2-9808d11d4c42\") " pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" Jul 07 15:04:43.051404 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.051217 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/89ce2379-8dcb-4fde-8423-47dc9bf4512f-snapshots\") pod \"insights-operator-5c847dcfc4-7qqkl\" (UID: \"89ce2379-8dcb-4fde-8423-47dc9bf4512f\") " pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" Jul 07 15:04:43.051404 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.051249 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkt46\" (UniqueName: \"kubernetes.io/projected/4faf5ae7-9e9a-41ac-aad2-9808d11d4c42-kube-api-access-vkt46\") pod \"console-operator-68b757fb67-z4h8q\" (UID: \"4faf5ae7-9e9a-41ac-aad2-9808d11d4c42\") " pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" Jul 07 15:04:43.051404 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.051289 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89ce2379-8dcb-4fde-8423-47dc9bf4512f-tmp\") pod \"insights-operator-5c847dcfc4-7qqkl\" (UID: \"89ce2379-8dcb-4fde-8423-47dc9bf4512f\") " pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" Jul 07 15:04:43.051404 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.051341 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/89ce2379-8dcb-4fde-8423-47dc9bf4512f-serving-cert\") pod \"insights-operator-5c847dcfc4-7qqkl\" (UID: \"89ce2379-8dcb-4fde-8423-47dc9bf4512f\") " pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" Jul 07 15:04:43.051404 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.051367 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-default-certificate\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:43.051404 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.051383 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq5mq\" (UniqueName: \"kubernetes.io/projected/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-kube-api-access-qq5mq\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:43.051659 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.051412 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4faf5ae7-9e9a-41ac-aad2-9808d11d4c42-serving-cert\") pod \"console-operator-68b757fb67-z4h8q\" (UID: \"4faf5ae7-9e9a-41ac-aad2-9808d11d4c42\") " pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" Jul 07 15:04:43.051659 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.051454 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-stats-auth\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:43.051659 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.051508 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggztb\" (UniqueName: \"kubernetes.io/projected/89ce2379-8dcb-4fde-8423-47dc9bf4512f-kube-api-access-ggztb\") pod \"insights-operator-5c847dcfc4-7qqkl\" (UID: \"89ce2379-8dcb-4fde-8423-47dc9bf4512f\") " pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" Jul 07 15:04:43.051659 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.051556 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89ce2379-8dcb-4fde-8423-47dc9bf4512f-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-7qqkl\" (UID: \"89ce2379-8dcb-4fde-8423-47dc9bf4512f\") " pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" Jul 07 15:04:43.074925 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.074852 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d"] Jul 07 15:04:43.077281 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.077264 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-zh2m8"] Jul 07 15:04:43.077411 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.077395 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d" Jul 07 15:04:43.079045 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.079024 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zh2m8" Jul 07 15:04:43.079170 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.079063 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wc9cn"] Jul 07 15:04:43.079476 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.079445 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 07 15:04:43.079694 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.079672 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 15:04:43.079801 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.079672 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kllbm\"" Jul 07 15:04:43.079801 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.079767 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 15:04:43.080678 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.080665 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 07 15:04:43.080860 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.080847 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wc9cn" Jul 07 15:04:43.082637 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.082618 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-j7xll\"" Jul 07 15:04:43.084601 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.084583 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 07 15:04:43.084890 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.084855 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 07 15:04:43.084958 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.084860 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 07 15:04:43.084958 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.084929 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:04:43.085048 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.085008 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-cx86b\"" Jul 07 15:04:43.087914 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.087894 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-zh2m8"] Jul 07 15:04:43.093663 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.093614 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d"] Jul 07 15:04:43.097906 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.097880 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wc9cn"] Jul 07 15:04:43.152527 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.152493 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4faf5ae7-9e9a-41ac-aad2-9808d11d4c42-config\") pod \"console-operator-68b757fb67-z4h8q\" (UID: \"4faf5ae7-9e9a-41ac-aad2-9808d11d4c42\") " pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" Jul 07 15:04:43.152527 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.152531 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8ldv\" (UniqueName: \"kubernetes.io/projected/414ac6cb-3670-4307-926a-9b6c9243107e-kube-api-access-h8ldv\") pod \"network-check-source-989f8b8dc-zh2m8\" (UID: \"414ac6cb-3670-4307-926a-9b6c9243107e\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zh2m8" Jul 07 15:04:43.152772 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.152566 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/89ce2379-8dcb-4fde-8423-47dc9bf4512f-serving-cert\") pod \"insights-operator-5c847dcfc4-7qqkl\" (UID: \"89ce2379-8dcb-4fde-8423-47dc9bf4512f\") " pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" Jul 07 15:04:43.152772 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.152594 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-default-certificate\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:43.152772 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.152620 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq5mq\" (UniqueName: \"kubernetes.io/projected/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-kube-api-access-qq5mq\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:43.152772 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.152650 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-q9n5d\" (UID: \"cc9cdf22-7b80-4c4e-941a-305d93a9ea4d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d" Jul 07 15:04:43.152772 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.152686 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/21e316ac-2667-4c4e-b726-88a3c866eb77-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wc9cn\" (UID: \"21e316ac-2667-4c4e-b726-88a3c866eb77\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wc9cn" Jul 07 15:04:43.152772 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.152732 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89ce2379-8dcb-4fde-8423-47dc9bf4512f-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-7qqkl\" (UID: \"89ce2379-8dcb-4fde-8423-47dc9bf4512f\") " pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" Jul 07 15:04:43.152772 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.152758 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/21e316ac-2667-4c4e-b726-88a3c866eb77-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wc9cn\" (UID: \"21e316ac-2667-4c4e-b726-88a3c866eb77\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wc9cn" Jul 07 15:04:43.153148 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.152797 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89ce2379-8dcb-4fde-8423-47dc9bf4512f-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-7qqkl\" (UID: \"89ce2379-8dcb-4fde-8423-47dc9bf4512f\") " pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" Jul 07 15:04:43.153148 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.152823 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-metrics-certs\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:43.153148 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.152851 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzdf8\" (UniqueName: \"kubernetes.io/projected/21e316ac-2667-4c4e-b726-88a3c866eb77-kube-api-access-jzdf8\") pod \"kube-storage-version-migrator-operator-846bf668f7-wc9cn\" (UID: \"21e316ac-2667-4c4e-b726-88a3c866eb77\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wc9cn" Jul 07 15:04:43.153148 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.152904 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/89ce2379-8dcb-4fde-8423-47dc9bf4512f-snapshots\") pod \"insights-operator-5c847dcfc4-7qqkl\" (UID: \"89ce2379-8dcb-4fde-8423-47dc9bf4512f\") " pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" Jul 07 15:04:43.153148 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.152932 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkt46\" (UniqueName: \"kubernetes.io/projected/4faf5ae7-9e9a-41ac-aad2-9808d11d4c42-kube-api-access-vkt46\") pod \"console-operator-68b757fb67-z4h8q\" (UID: \"4faf5ae7-9e9a-41ac-aad2-9808d11d4c42\") " pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" Jul 07 15:04:43.153148 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.152963 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89ce2379-8dcb-4fde-8423-47dc9bf4512f-tmp\") pod \"insights-operator-5c847dcfc4-7qqkl\" (UID: \"89ce2379-8dcb-4fde-8423-47dc9bf4512f\") " pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" Jul 07 15:04:43.153148 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.153008 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl9mj\" (UniqueName: \"kubernetes.io/projected/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-kube-api-access-pl9mj\") pod \"cluster-monitoring-operator-64786d9cbb-q9n5d\" (UID: \"cc9cdf22-7b80-4c4e-941a-305d93a9ea4d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d" Jul 07 15:04:43.153148 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.153069 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4faf5ae7-9e9a-41ac-aad2-9808d11d4c42-serving-cert\") pod \"console-operator-68b757fb67-z4h8q\" (UID: \"4faf5ae7-9e9a-41ac-aad2-9808d11d4c42\") " pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" Jul 07 15:04:43.153148 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.153127 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-stats-auth\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:43.154061 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.153154 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-q9n5d\" (UID: \"cc9cdf22-7b80-4c4e-941a-305d93a9ea4d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d" Jul 07 15:04:43.154061 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.153190 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggztb\" (UniqueName: \"kubernetes.io/projected/89ce2379-8dcb-4fde-8423-47dc9bf4512f-kube-api-access-ggztb\") pod \"insights-operator-5c847dcfc4-7qqkl\" (UID: \"89ce2379-8dcb-4fde-8423-47dc9bf4512f\") " pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" Jul 07 15:04:43.154061 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.153222 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-service-ca-bundle\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:43.154061 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.153248 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4faf5ae7-9e9a-41ac-aad2-9808d11d4c42-trusted-ca\") pod \"console-operator-68b757fb67-z4h8q\" (UID: \"4faf5ae7-9e9a-41ac-aad2-9808d11d4c42\") " pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" Jul 07 15:04:43.154061 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.153265 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/89ce2379-8dcb-4fde-8423-47dc9bf4512f-tmp\") pod \"insights-operator-5c847dcfc4-7qqkl\" (UID: \"89ce2379-8dcb-4fde-8423-47dc9bf4512f\") " pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" Jul 07 15:04:43.154061 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.153369 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4faf5ae7-9e9a-41ac-aad2-9808d11d4c42-config\") pod \"console-operator-68b757fb67-z4h8q\" (UID: \"4faf5ae7-9e9a-41ac-aad2-9808d11d4c42\") " pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" Jul 07 15:04:43.154061 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:43.153533 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-service-ca-bundle podName:0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7 nodeName:}" failed. No retries permitted until 2026-07-07 15:04:43.653513115 +0000 UTC m=+232.469779148 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-service-ca-bundle") pod "router-default-5c76867bd5-72zmj" (UID: "0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7") : configmap references non-existent config key: service-ca.crt Jul 07 15:04:43.154061 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:43.153552 2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 15:04:43.154061 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:43.153590 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-metrics-certs podName:0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7 nodeName:}" failed. No retries permitted until 2026-07-07 15:04:43.653578683 +0000 UTC m=+232.469844700 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-metrics-certs") pod "router-default-5c76867bd5-72zmj" (UID: "0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7") : secret "router-metrics-certs-default" not found Jul 07 15:04:43.154061 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.153946 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89ce2379-8dcb-4fde-8423-47dc9bf4512f-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-7qqkl\" (UID: \"89ce2379-8dcb-4fde-8423-47dc9bf4512f\") " pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" Jul 07 15:04:43.154061 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.153999 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89ce2379-8dcb-4fde-8423-47dc9bf4512f-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-7qqkl\" (UID: \"89ce2379-8dcb-4fde-8423-47dc9bf4512f\") " pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" Jul 07 15:04:43.154061 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.154055 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/89ce2379-8dcb-4fde-8423-47dc9bf4512f-snapshots\") pod \"insights-operator-5c847dcfc4-7qqkl\" (UID: \"89ce2379-8dcb-4fde-8423-47dc9bf4512f\") " pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" Jul 07 15:04:43.154504 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.154239 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4faf5ae7-9e9a-41ac-aad2-9808d11d4c42-trusted-ca\") pod \"console-operator-68b757fb67-z4h8q\" (UID: \"4faf5ae7-9e9a-41ac-aad2-9808d11d4c42\") " pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" Jul 07 15:04:43.155462 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.155435 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-default-certificate\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:43.155561 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.155528 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/89ce2379-8dcb-4fde-8423-47dc9bf4512f-serving-cert\") pod \"insights-operator-5c847dcfc4-7qqkl\" (UID: \"89ce2379-8dcb-4fde-8423-47dc9bf4512f\") " pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" Jul 07 15:04:43.155752 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.155734 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4faf5ae7-9e9a-41ac-aad2-9808d11d4c42-serving-cert\") pod \"console-operator-68b757fb67-z4h8q\" (UID: \"4faf5ae7-9e9a-41ac-aad2-9808d11d4c42\") " pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" Jul 07 15:04:43.156151 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.156136 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-stats-auth\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:43.161928 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.161907 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq5mq\" (UniqueName: \"kubernetes.io/projected/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-kube-api-access-qq5mq\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:43.162266 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.162245 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkt46\" (UniqueName: \"kubernetes.io/projected/4faf5ae7-9e9a-41ac-aad2-9808d11d4c42-kube-api-access-vkt46\") pod \"console-operator-68b757fb67-z4h8q\" (UID: \"4faf5ae7-9e9a-41ac-aad2-9808d11d4c42\") " pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" Jul 07 15:04:43.162340 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.162315 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggztb\" (UniqueName: \"kubernetes.io/projected/89ce2379-8dcb-4fde-8423-47dc9bf4512f-kube-api-access-ggztb\") pod \"insights-operator-5c847dcfc4-7qqkl\" (UID: \"89ce2379-8dcb-4fde-8423-47dc9bf4512f\") " pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" Jul 07 15:04:43.254612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.254511 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzdf8\" (UniqueName: \"kubernetes.io/projected/21e316ac-2667-4c4e-b726-88a3c866eb77-kube-api-access-jzdf8\") pod \"kube-storage-version-migrator-operator-846bf668f7-wc9cn\" (UID: \"21e316ac-2667-4c4e-b726-88a3c866eb77\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wc9cn" Jul 07 15:04:43.254612 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.254566 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl9mj\" (UniqueName: \"kubernetes.io/projected/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-kube-api-access-pl9mj\") pod \"cluster-monitoring-operator-64786d9cbb-q9n5d\" (UID: \"cc9cdf22-7b80-4c4e-941a-305d93a9ea4d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d" Jul 07 15:04:43.254950 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.254616 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-q9n5d\" (UID: \"cc9cdf22-7b80-4c4e-941a-305d93a9ea4d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d" Jul 07 15:04:43.254950 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.254650 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8ldv\" (UniqueName: \"kubernetes.io/projected/414ac6cb-3670-4307-926a-9b6c9243107e-kube-api-access-h8ldv\") pod \"network-check-source-989f8b8dc-zh2m8\" (UID: \"414ac6cb-3670-4307-926a-9b6c9243107e\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zh2m8" Jul 07 15:04:43.254950 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.254672 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-q9n5d\" (UID: \"cc9cdf22-7b80-4c4e-941a-305d93a9ea4d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d" Jul 07 15:04:43.254950 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.254700 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/21e316ac-2667-4c4e-b726-88a3c866eb77-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wc9cn\" (UID: \"21e316ac-2667-4c4e-b726-88a3c866eb77\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wc9cn" Jul 07 15:04:43.254950 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:43.254803 2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 15:04:43.254950 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.254795 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/21e316ac-2667-4c4e-b726-88a3c866eb77-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wc9cn\" (UID: \"21e316ac-2667-4c4e-b726-88a3c866eb77\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wc9cn" Jul 07 15:04:43.254950 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:43.254915 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-cluster-monitoring-operator-tls podName:cc9cdf22-7b80-4c4e-941a-305d93a9ea4d nodeName:}" failed. No retries permitted until 2026-07-07 15:04:43.754891694 +0000 UTC m=+232.571157726 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-q9n5d" (UID: "cc9cdf22-7b80-4c4e-941a-305d93a9ea4d") : secret "cluster-monitoring-operator-tls" not found Jul 07 15:04:43.255371 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.255349 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/21e316ac-2667-4c4e-b726-88a3c866eb77-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wc9cn\" (UID: \"21e316ac-2667-4c4e-b726-88a3c866eb77\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wc9cn" Jul 07 15:04:43.255533 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.255510 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-q9n5d\" (UID: \"cc9cdf22-7b80-4c4e-941a-305d93a9ea4d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d" Jul 07 15:04:43.257030 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.257011 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/21e316ac-2667-4c4e-b726-88a3c866eb77-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wc9cn\" (UID: \"21e316ac-2667-4c4e-b726-88a3c866eb77\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wc9cn" Jul 07 15:04:43.265942 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.265917 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl9mj\" (UniqueName: \"kubernetes.io/projected/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-kube-api-access-pl9mj\") pod \"cluster-monitoring-operator-64786d9cbb-q9n5d\" (UID: \"cc9cdf22-7b80-4c4e-941a-305d93a9ea4d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d" Jul 07 15:04:43.266066 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.266013 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzdf8\" (UniqueName: \"kubernetes.io/projected/21e316ac-2667-4c4e-b726-88a3c866eb77-kube-api-access-jzdf8\") pod \"kube-storage-version-migrator-operator-846bf668f7-wc9cn\" (UID: \"21e316ac-2667-4c4e-b726-88a3c866eb77\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wc9cn" Jul 07 15:04:43.266946 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.266923 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8ldv\" (UniqueName: \"kubernetes.io/projected/414ac6cb-3670-4307-926a-9b6c9243107e-kube-api-access-h8ldv\") pod \"network-check-source-989f8b8dc-zh2m8\" (UID: \"414ac6cb-3670-4307-926a-9b6c9243107e\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zh2m8" Jul 07 15:04:43.284903 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.284853 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" Jul 07 15:04:43.294076 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.294046 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" Jul 07 15:04:43.394125 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.394096 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zh2m8" Jul 07 15:04:43.400434 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.400407 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wc9cn" Jul 07 15:04:43.426433 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.426406 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-7qqkl"] Jul 07 15:04:43.429834 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:04:43.429802 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89ce2379_8dcb_4fde_8423_47dc9bf4512f.slice/crio-4aa7b43bbadb470261ce991b208a7ae3b614c073c9a7a81583f1c4220a898b08 WatchSource:0}: Error finding container 4aa7b43bbadb470261ce991b208a7ae3b614c073c9a7a81583f1c4220a898b08: Status 404 returned error can't find the container with id 4aa7b43bbadb470261ce991b208a7ae3b614c073c9a7a81583f1c4220a898b08 Jul 07 15:04:43.447381 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.447304 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-z4h8q"] Jul 07 15:04:43.453539 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:04:43.453494 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4faf5ae7_9e9a_41ac_aad2_9808d11d4c42.slice/crio-94aaa4156d12c3f3747a178940de0cef578f3500bcd09ca63b1a421461f8a04c WatchSource:0}: Error finding container 94aaa4156d12c3f3747a178940de0cef578f3500bcd09ca63b1a421461f8a04c: Status 404 returned error can't find the container with id 94aaa4156d12c3f3747a178940de0cef578f3500bcd09ca63b1a421461f8a04c Jul 07 15:04:43.471495 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.471428 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" event={"ID":"89ce2379-8dcb-4fde-8423-47dc9bf4512f","Type":"ContainerStarted","Data":"4aa7b43bbadb470261ce991b208a7ae3b614c073c9a7a81583f1c4220a898b08"} Jul 07 15:04:43.472618 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.472594 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" event={"ID":"4faf5ae7-9e9a-41ac-aad2-9808d11d4c42","Type":"ContainerStarted","Data":"94aaa4156d12c3f3747a178940de0cef578f3500bcd09ca63b1a421461f8a04c"} Jul 07 15:04:43.533092 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.532995 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wc9cn"] Jul 07 15:04:43.535653 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:04:43.535620 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21e316ac_2667_4c4e_b726_88a3c866eb77.slice/crio-a82753197e3eba09a7350562fdd666d92500045ba69239fb147f56db88e0c5f1 WatchSource:0}: Error finding container a82753197e3eba09a7350562fdd666d92500045ba69239fb147f56db88e0c5f1: Status 404 returned error can't find the container with id a82753197e3eba09a7350562fdd666d92500045ba69239fb147f56db88e0c5f1 Jul 07 15:04:43.545540 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.545515 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-zh2m8"] Jul 07 15:04:43.548385 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:04:43.548362 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod414ac6cb_3670_4307_926a_9b6c9243107e.slice/crio-4de81d03c365b4969c41e01e24bc17f354b8b749e4868d7ede38d40475095136 WatchSource:0}: Error finding container 4de81d03c365b4969c41e01e24bc17f354b8b749e4868d7ede38d40475095136: Status 404 returned error can't find the container with id 4de81d03c365b4969c41e01e24bc17f354b8b749e4868d7ede38d40475095136 Jul 07 15:04:43.658135 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.658098 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-metrics-certs\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:43.658395 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.658167 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-service-ca-bundle\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:43.658395 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:43.658258 2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 15:04:43.658395 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:43.658309 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-service-ca-bundle podName:0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7 nodeName:}" failed. No retries permitted until 2026-07-07 15:04:44.658291599 +0000 UTC m=+233.474557616 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-service-ca-bundle") pod "router-default-5c76867bd5-72zmj" (UID: "0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7") : configmap references non-existent config key: service-ca.crt Jul 07 15:04:43.658395 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:43.658330 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-metrics-certs podName:0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7 nodeName:}" failed. No retries permitted until 2026-07-07 15:04:44.658320795 +0000 UTC m=+233.474586813 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-metrics-certs") pod "router-default-5c76867bd5-72zmj" (UID: "0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7") : secret "router-metrics-certs-default" not found Jul 07 15:04:43.759076 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:43.759027 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-q9n5d\" (UID: \"cc9cdf22-7b80-4c4e-941a-305d93a9ea4d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d" Jul 07 15:04:43.759220 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:43.759169 2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 15:04:43.759279 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:43.759230 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-cluster-monitoring-operator-tls podName:cc9cdf22-7b80-4c4e-941a-305d93a9ea4d nodeName:}" failed. No retries permitted until 2026-07-07 15:04:44.759215132 +0000 UTC m=+233.575481156 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-q9n5d" (UID: "cc9cdf22-7b80-4c4e-941a-305d93a9ea4d") : secret "cluster-monitoring-operator-tls" not found Jul 07 15:04:44.479208 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:44.478470 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zh2m8" event={"ID":"414ac6cb-3670-4307-926a-9b6c9243107e","Type":"ContainerStarted","Data":"425a2ae9add61e041232883f839fec8f2b1c98dfce2ea6dbd9b5d9c824a85ef8"} Jul 07 15:04:44.479208 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:44.478533 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zh2m8" event={"ID":"414ac6cb-3670-4307-926a-9b6c9243107e","Type":"ContainerStarted","Data":"4de81d03c365b4969c41e01e24bc17f354b8b749e4868d7ede38d40475095136"} Jul 07 15:04:44.481492 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:44.481393 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wc9cn" event={"ID":"21e316ac-2667-4c4e-b726-88a3c866eb77","Type":"ContainerStarted","Data":"a82753197e3eba09a7350562fdd666d92500045ba69239fb147f56db88e0c5f1"} Jul 07 15:04:44.496309 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:44.496081 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-zh2m8" podStartSLOduration=1.496058808 podStartE2EDuration="1.496058808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:04:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:04:44.494185236 +0000 UTC m=+233.310451278" watchObservedRunningTime="2026-07-07 15:04:44.496058808 +0000 UTC m=+233.312324850" Jul 07 15:04:44.667730 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:44.667690 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-metrics-certs\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:44.667924 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:44.667779 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-service-ca-bundle\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:44.667924 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:44.667862 2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 15:04:44.668044 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:44.667933 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-service-ca-bundle podName:0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7 nodeName:}" failed. No retries permitted until 2026-07-07 15:04:46.667912654 +0000 UTC m=+235.484178677 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-service-ca-bundle") pod "router-default-5c76867bd5-72zmj" (UID: "0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7") : configmap references non-existent config key: service-ca.crt Jul 07 15:04:44.668044 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:44.667954 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-metrics-certs podName:0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7 nodeName:}" failed. No retries permitted until 2026-07-07 15:04:46.667945148 +0000 UTC m=+235.484211166 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-metrics-certs") pod "router-default-5c76867bd5-72zmj" (UID: "0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7") : secret "router-metrics-certs-default" not found Jul 07 15:04:44.768843 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:44.768759 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-q9n5d\" (UID: \"cc9cdf22-7b80-4c4e-941a-305d93a9ea4d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d" Jul 07 15:04:44.769015 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:44.768944 2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 15:04:44.769156 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:44.769019 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-cluster-monitoring-operator-tls podName:cc9cdf22-7b80-4c4e-941a-305d93a9ea4d nodeName:}" failed. No retries permitted until 2026-07-07 15:04:46.768998022 +0000 UTC m=+235.585264055 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-q9n5d" (UID: "cc9cdf22-7b80-4c4e-941a-305d93a9ea4d") : secret "cluster-monitoring-operator-tls" not found Jul 07 15:04:46.682786 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:46.682744 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-service-ca-bundle\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:46.683288 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:46.682953 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-metrics-certs\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:46.683288 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:46.683026 2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 15:04:46.683288 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:46.683030 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-service-ca-bundle podName:0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7 nodeName:}" failed. No retries permitted until 2026-07-07 15:04:50.683008384 +0000 UTC m=+239.499274407 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-service-ca-bundle") pod "router-default-5c76867bd5-72zmj" (UID: "0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7") : configmap references non-existent config key: service-ca.crt Jul 07 15:04:46.683288 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:46.683080 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-metrics-certs podName:0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7 nodeName:}" failed. No retries permitted until 2026-07-07 15:04:50.683067065 +0000 UTC m=+239.499333082 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-metrics-certs") pod "router-default-5c76867bd5-72zmj" (UID: "0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7") : secret "router-metrics-certs-default" not found Jul 07 15:04:46.783463 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:46.783419 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-q9n5d\" (UID: \"cc9cdf22-7b80-4c4e-941a-305d93a9ea4d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d" Jul 07 15:04:46.783629 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:46.783555 2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 15:04:46.783681 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:46.783652 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-cluster-monitoring-operator-tls podName:cc9cdf22-7b80-4c4e-941a-305d93a9ea4d nodeName:}" failed. No retries permitted until 2026-07-07 15:04:50.783630245 +0000 UTC m=+239.599896282 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-q9n5d" (UID: "cc9cdf22-7b80-4c4e-941a-305d93a9ea4d") : secret "cluster-monitoring-operator-tls" not found Jul 07 15:04:47.491361 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:47.491321 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" event={"ID":"89ce2379-8dcb-4fde-8423-47dc9bf4512f","Type":"ContainerStarted","Data":"d9cf85011ec339d313076a680773c9943e305b38acc7d90cdef96e4603211484"} Jul 07 15:04:47.492899 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:47.492851 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wc9cn" event={"ID":"21e316ac-2667-4c4e-b726-88a3c866eb77","Type":"ContainerStarted","Data":"f240fa6c9419318ace4293cf831207ea6413b1d2c5a05a88ec6070ac3ca6de71"} Jul 07 15:04:47.494484 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:47.494462 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/0.log" Jul 07 15:04:47.494628 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:47.494499 2556 generic.go:356] "Generic (PLEG): container finished" podID="4faf5ae7-9e9a-41ac-aad2-9808d11d4c42" containerID="3cf047d8a6de5e47337467d9f37a6a1200553e1cddf186569434ce1f633663db" exitCode=255 Jul 07 15:04:47.494628 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:47.494528 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" event={"ID":"4faf5ae7-9e9a-41ac-aad2-9808d11d4c42","Type":"ContainerDied","Data":"3cf047d8a6de5e47337467d9f37a6a1200553e1cddf186569434ce1f633663db"} Jul 07 15:04:47.494739 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:47.494723 2556 scope.go:117] "RemoveContainer" containerID="3cf047d8a6de5e47337467d9f37a6a1200553e1cddf186569434ce1f633663db" Jul 07 15:04:47.508077 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:47.508028 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" podStartSLOduration=1.9911507080000002 podStartE2EDuration="5.508009321s" totalImagesPullingTime="3.516858613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:04:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:04:47.50747912 +0000 UTC m=+236.323745158" watchObservedRunningTime="2026-07-07 15:04:47.508009321 +0000 UTC m=+236.324275361" Jul 07 15:04:47.539537 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:47.539486 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wc9cn" podStartSLOduration=1.121083241 podStartE2EDuration="4.539468787s" totalImagesPullingTime="3.418385546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:04:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:04:47.538772976 +0000 UTC m=+236.355039018" watchObservedRunningTime="2026-07-07 15:04:47.539468787 +0000 UTC m=+236.355734826" Jul 07 15:04:48.119796 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.119765 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7lnj4"] Jul 07 15:04:48.121998 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.121978 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:04:48.124656 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.124632 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 15:04:48.124656 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.124642 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gkfhx\"" Jul 07 15:04:48.124811 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.124785 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 15:04:48.136028 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.136000 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7lnj4"] Jul 07 15:04:48.195160 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.195128 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ghdt\" (UniqueName: \"kubernetes.io/projected/f1529a40-d0ce-481c-8233-200713ff5d42-kube-api-access-9ghdt\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:04:48.195306 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.195180 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1529a40-d0ce-481c-8233-200713ff5d42-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:04:48.195306 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.195205 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f1529a40-d0ce-481c-8233-200713ff5d42-crio-socket\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:04:48.195306 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.195260 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1529a40-d0ce-481c-8233-200713ff5d42-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:04:48.195306 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.195279 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1529a40-d0ce-481c-8233-200713ff5d42-data-volume\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:04:48.296186 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.296087 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1529a40-d0ce-481c-8233-200713ff5d42-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:04:48.296186 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.296136 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f1529a40-d0ce-481c-8233-200713ff5d42-crio-socket\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:04:48.296186 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.296189 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1529a40-d0ce-481c-8233-200713ff5d42-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:04:48.296493 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.296213 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1529a40-d0ce-481c-8233-200713ff5d42-data-volume\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:04:48.296493 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:48.296231 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 15:04:48.296493 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:48.296302 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 15:04:48.296493 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.296297 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f1529a40-d0ce-481c-8233-200713ff5d42-crio-socket\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:04:48.296493 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:48.296355 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1529a40-d0ce-481c-8233-200713ff5d42-kube-rbac-proxy-cm podName:f1529a40-d0ce-481c-8233-200713ff5d42 nodeName:}" failed. No retries permitted until 2026-07-07 15:04:48.796336457 +0000 UTC m=+237.612602475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f1529a40-d0ce-481c-8233-200713ff5d42-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7lnj4" (UID: "f1529a40-d0ce-481c-8233-200713ff5d42") : configmap "kube-rbac-proxy" not found Jul 07 15:04:48.296493 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:48.296381 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1529a40-d0ce-481c-8233-200713ff5d42-insights-runtime-extractor-tls podName:f1529a40-d0ce-481c-8233-200713ff5d42 nodeName:}" failed. No retries permitted until 2026-07-07 15:04:48.796374748 +0000 UTC m=+237.612640765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1529a40-d0ce-481c-8233-200713ff5d42-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7lnj4" (UID: "f1529a40-d0ce-481c-8233-200713ff5d42") : secret "insights-runtime-extractor-tls" not found Jul 07 15:04:48.296493 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.296411 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ghdt\" (UniqueName: \"kubernetes.io/projected/f1529a40-d0ce-481c-8233-200713ff5d42-kube-api-access-9ghdt\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:04:48.296764 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.296554 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1529a40-d0ce-481c-8233-200713ff5d42-data-volume\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:04:48.304921 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.304895 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ghdt\" (UniqueName: \"kubernetes.io/projected/f1529a40-d0ce-481c-8233-200713ff5d42-kube-api-access-9ghdt\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:04:48.499284 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.499259 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/1.log" Jul 07 15:04:48.499700 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.499685 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/0.log" Jul 07 15:04:48.499748 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.499720 2556 generic.go:356] "Generic (PLEG): container finished" podID="4faf5ae7-9e9a-41ac-aad2-9808d11d4c42" containerID="e4904cca71e3250cd6c8f3586d1da589f8507d6f3627e4e339362d0d54826021" exitCode=255 Jul 07 15:04:48.499809 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.499752 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" event={"ID":"4faf5ae7-9e9a-41ac-aad2-9808d11d4c42","Type":"ContainerDied","Data":"e4904cca71e3250cd6c8f3586d1da589f8507d6f3627e4e339362d0d54826021"} Jul 07 15:04:48.499863 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.499847 2556 scope.go:117] "RemoveContainer" containerID="3cf047d8a6de5e47337467d9f37a6a1200553e1cddf186569434ce1f633663db" Jul 07 15:04:48.500116 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.500093 2556 scope.go:117] "RemoveContainer" containerID="e4904cca71e3250cd6c8f3586d1da589f8507d6f3627e4e339362d0d54826021" Jul 07 15:04:48.502480 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:48.500554 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-z4h8q_openshift-console-operator(4faf5ae7-9e9a-41ac-aad2-9808d11d4c42)\"" pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" podUID="4faf5ae7-9e9a-41ac-aad2-9808d11d4c42" Jul 07 15:04:48.801751 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.801711 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1529a40-d0ce-481c-8233-200713ff5d42-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:04:48.801995 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:48.801765 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1529a40-d0ce-481c-8233-200713ff5d42-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:04:48.801995 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:48.801835 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 15:04:48.801995 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:48.801903 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1529a40-d0ce-481c-8233-200713ff5d42-kube-rbac-proxy-cm podName:f1529a40-d0ce-481c-8233-200713ff5d42 nodeName:}" failed. No retries permitted until 2026-07-07 15:04:49.801888513 +0000 UTC m=+238.618154531 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f1529a40-d0ce-481c-8233-200713ff5d42-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7lnj4" (UID: "f1529a40-d0ce-481c-8233-200713ff5d42") : configmap "kube-rbac-proxy" not found Jul 07 15:04:48.801995 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:48.801912 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 15:04:48.802165 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:48.802000 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1529a40-d0ce-481c-8233-200713ff5d42-insights-runtime-extractor-tls podName:f1529a40-d0ce-481c-8233-200713ff5d42 nodeName:}" failed. No retries permitted until 2026-07-07 15:04:49.801967326 +0000 UTC m=+238.618233350 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1529a40-d0ce-481c-8233-200713ff5d42-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7lnj4" (UID: "f1529a40-d0ce-481c-8233-200713ff5d42") : secret "insights-runtime-extractor-tls" not found Jul 07 15:04:49.504213 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:49.504185 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/1.log" Jul 07 15:04:49.504610 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:49.504499 2556 scope.go:117] "RemoveContainer" containerID="e4904cca71e3250cd6c8f3586d1da589f8507d6f3627e4e339362d0d54826021" Jul 07 15:04:49.504659 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:49.504638 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-z4h8q_openshift-console-operator(4faf5ae7-9e9a-41ac-aad2-9808d11d4c42)\"" pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" podUID="4faf5ae7-9e9a-41ac-aad2-9808d11d4c42" Jul 07 15:04:49.810329 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:49.810225 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1529a40-d0ce-481c-8233-200713ff5d42-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:04:49.810329 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:49.810301 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1529a40-d0ce-481c-8233-200713ff5d42-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:04:49.810546 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:49.810375 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 15:04:49.810546 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:49.810422 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 15:04:49.810546 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:49.810436 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1529a40-d0ce-481c-8233-200713ff5d42-insights-runtime-extractor-tls podName:f1529a40-d0ce-481c-8233-200713ff5d42 nodeName:}" failed. No retries permitted until 2026-07-07 15:04:51.810421049 +0000 UTC m=+240.626687066 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1529a40-d0ce-481c-8233-200713ff5d42-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7lnj4" (UID: "f1529a40-d0ce-481c-8233-200713ff5d42") : secret "insights-runtime-extractor-tls" not found Jul 07 15:04:49.810546 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:49.810482 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1529a40-d0ce-481c-8233-200713ff5d42-kube-rbac-proxy-cm podName:f1529a40-d0ce-481c-8233-200713ff5d42 nodeName:}" failed. No retries permitted until 2026-07-07 15:04:51.810468737 +0000 UTC m=+240.626734755 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f1529a40-d0ce-481c-8233-200713ff5d42-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7lnj4" (UID: "f1529a40-d0ce-481c-8233-200713ff5d42") : configmap "kube-rbac-proxy" not found Jul 07 15:04:50.716256 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:50.716217 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-service-ca-bundle\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:50.716641 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:50.716325 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-metrics-certs\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:50.716641 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:50.716406 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-service-ca-bundle podName:0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7 nodeName:}" failed. No retries permitted until 2026-07-07 15:04:58.71638679 +0000 UTC m=+247.532652820 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-service-ca-bundle") pod "router-default-5c76867bd5-72zmj" (UID: "0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7") : configmap references non-existent config key: service-ca.crt Jul 07 15:04:50.716641 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:50.716441 2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 15:04:50.716641 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:50.716478 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-metrics-certs podName:0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7 nodeName:}" failed. No retries permitted until 2026-07-07 15:04:58.716466897 +0000 UTC m=+247.532732914 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-metrics-certs") pod "router-default-5c76867bd5-72zmj" (UID: "0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7") : secret "router-metrics-certs-default" not found Jul 07 15:04:50.816776 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:50.816730 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-q9n5d\" (UID: \"cc9cdf22-7b80-4c4e-941a-305d93a9ea4d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d" Jul 07 15:04:50.816991 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:50.816891 2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 15:04:50.816991 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:50.816962 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-cluster-monitoring-operator-tls podName:cc9cdf22-7b80-4c4e-941a-305d93a9ea4d nodeName:}" failed. No retries permitted until 2026-07-07 15:04:58.816942674 +0000 UTC m=+247.633208694 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-q9n5d" (UID: "cc9cdf22-7b80-4c4e-941a-305d93a9ea4d") : secret "cluster-monitoring-operator-tls" not found Jul 07 15:04:51.101540 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:51.101506 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-spm2h_31976396-202a-47d9-b31f-9bf2efdfd835/dns-node-resolver/0.log" Jul 07 15:04:51.825480 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:51.825427 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1529a40-d0ce-481c-8233-200713ff5d42-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:04:51.825915 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:51.825573 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 15:04:51.825915 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:51.825629 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1529a40-d0ce-481c-8233-200713ff5d42-insights-runtime-extractor-tls podName:f1529a40-d0ce-481c-8233-200713ff5d42 nodeName:}" failed. No retries permitted until 2026-07-07 15:04:55.825614498 +0000 UTC m=+244.641880516 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1529a40-d0ce-481c-8233-200713ff5d42-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7lnj4" (UID: "f1529a40-d0ce-481c-8233-200713ff5d42") : secret "insights-runtime-extractor-tls" not found Jul 07 15:04:51.825915 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:51.825631 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 15:04:51.825915 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:51.825578 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1529a40-d0ce-481c-8233-200713ff5d42-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:04:51.825915 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:51.825682 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1529a40-d0ce-481c-8233-200713ff5d42-kube-rbac-proxy-cm podName:f1529a40-d0ce-481c-8233-200713ff5d42 nodeName:}" failed. No retries permitted until 2026-07-07 15:04:55.825669675 +0000 UTC m=+244.641935691 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f1529a40-d0ce-481c-8233-200713ff5d42-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7lnj4" (UID: "f1529a40-d0ce-481c-8233-200713ff5d42") : configmap "kube-rbac-proxy" not found Jul 07 15:04:52.101003 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:52.100975 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w6w74_da182a28-bbef-4bfd-9273-b94f43794676/node-ca/0.log" Jul 07 15:04:52.502551 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:52.502472 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cm8wk_b040cb56-952e-4172-a484-89ba96e76a45/serve-healthcheck-canary/0.log" Jul 07 15:04:53.294623 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:53.294537 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" Jul 07 15:04:53.294623 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:53.294615 2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" Jul 07 15:04:53.295108 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:53.294967 2556 scope.go:117] "RemoveContainer" containerID="e4904cca71e3250cd6c8f3586d1da589f8507d6f3627e4e339362d0d54826021" Jul 07 15:04:53.295156 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:53.295137 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-z4h8q_openshift-console-operator(4faf5ae7-9e9a-41ac-aad2-9808d11d4c42)\"" pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" podUID="4faf5ae7-9e9a-41ac-aad2-9808d11d4c42" Jul 07 15:04:53.503103 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:53.503070 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-wc9cn_21e316ac-2667-4c4e-b726-88a3c866eb77/kube-storage-version-migrator-operator/0.log" Jul 07 15:04:55.856294 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:55.856258 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1529a40-d0ce-481c-8233-200713ff5d42-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:04:55.856728 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:55.856310 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1529a40-d0ce-481c-8233-200713ff5d42-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:04:55.856728 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:55.856409 2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 15:04:55.856728 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:55.856411 2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 15:04:55.856728 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:55.856469 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1529a40-d0ce-481c-8233-200713ff5d42-kube-rbac-proxy-cm podName:f1529a40-d0ce-481c-8233-200713ff5d42 nodeName:}" failed. No retries permitted until 2026-07-07 15:05:03.856454225 +0000 UTC m=+252.672720242 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f1529a40-d0ce-481c-8233-200713ff5d42-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7lnj4" (UID: "f1529a40-d0ce-481c-8233-200713ff5d42") : configmap "kube-rbac-proxy" not found Jul 07 15:04:55.856728 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:55.856483 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1529a40-d0ce-481c-8233-200713ff5d42-insights-runtime-extractor-tls podName:f1529a40-d0ce-481c-8233-200713ff5d42 nodeName:}" failed. No retries permitted until 2026-07-07 15:05:03.856476293 +0000 UTC m=+252.672742310 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1529a40-d0ce-481c-8233-200713ff5d42-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7lnj4" (UID: "f1529a40-d0ce-481c-8233-200713ff5d42") : secret "insights-runtime-extractor-tls" not found Jul 07 15:04:58.780145 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:58.780110 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-service-ca-bundle\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:58.780565 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:58.780295 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-metrics-certs\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:58.785139 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:58.781827 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-service-ca-bundle\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:58.785139 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:58.784584 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7-metrics-certs\") pod \"router-default-5c76867bd5-72zmj\" (UID: \"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7\") " pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:58.881130 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:58.881088 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-q9n5d\" (UID: \"cc9cdf22-7b80-4c4e-941a-305d93a9ea4d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d" Jul 07 15:04:58.881307 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:58.881291 2556 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 15:04:58.881381 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:04:58.881369 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-cluster-monitoring-operator-tls podName:cc9cdf22-7b80-4c4e-941a-305d93a9ea4d nodeName:}" failed. No retries permitted until 2026-07-07 15:05:14.88135301 +0000 UTC m=+263.697619031 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-q9n5d" (UID: "cc9cdf22-7b80-4c4e-941a-305d93a9ea4d") : secret "cluster-monitoring-operator-tls" not found Jul 07 15:04:58.900721 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:58.900690 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4pm6j\"" Jul 07 15:04:58.908616 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:58.908588 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:59.034597 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:59.034522 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5c76867bd5-72zmj"] Jul 07 15:04:59.037348 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:04:59.037318 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0952af4f_f3dd_4bdf_b0da_5c0d66fb57b7.slice/crio-686eb052cfadb4b426c7937dd0ae2c58214ba17125bd3bfdd40c6bf600051e7d WatchSource:0}: Error finding container 686eb052cfadb4b426c7937dd0ae2c58214ba17125bd3bfdd40c6bf600051e7d: Status 404 returned error can't find the container with id 686eb052cfadb4b426c7937dd0ae2c58214ba17125bd3bfdd40c6bf600051e7d Jul 07 15:04:59.536008 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:59.535968 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5c76867bd5-72zmj" event={"ID":"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7","Type":"ContainerStarted","Data":"59397b91fd933d6a621e563e5f64efb4e102ba64112a5184afb1430a22fce4ca"} Jul 07 15:04:59.536191 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:59.536031 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5c76867bd5-72zmj" event={"ID":"0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7","Type":"ContainerStarted","Data":"686eb052cfadb4b426c7937dd0ae2c58214ba17125bd3bfdd40c6bf600051e7d"} Jul 07 15:04:59.556196 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:59.556145 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5c76867bd5-72zmj" podStartSLOduration=17.556128326 podStartE2EDuration="17.556128326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:04:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:04:59.55581775 +0000 UTC m=+248.372083789" watchObservedRunningTime="2026-07-07 15:04:59.556128326 +0000 UTC m=+248.372394364" Jul 07 15:04:59.909763 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:59.909725 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:04:59.912380 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:04:59.912356 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:05:00.539720 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:00.539694 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:05:00.540844 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:00.540824 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5c76867bd5-72zmj" Jul 07 15:05:03.524029 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:03.523988 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs\") pod \"network-metrics-daemon-dq8k2\" (UID: \"f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a\") " pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:05:03.526254 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:03.526236 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a-metrics-certs\") pod \"network-metrics-daemon-dq8k2\" (UID: \"f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a\") " pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:05:03.684791 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:03.684761 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-mbwr6\"" Jul 07 15:05:03.692601 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:03.692583 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dq8k2" Jul 07 15:05:03.818204 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:03.815042 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dq8k2"] Jul 07 15:05:03.819486 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:05:03.819457 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6f81ab4_1ef1_4fdf_b6c7_50023a27d54a.slice/crio-0652b2fcd703ed4f717afde1d7d729eaa426d9c0e131c996039f6313d71ff7d6 WatchSource:0}: Error finding container 0652b2fcd703ed4f717afde1d7d729eaa426d9c0e131c996039f6313d71ff7d6: Status 404 returned error can't find the container with id 0652b2fcd703ed4f717afde1d7d729eaa426d9c0e131c996039f6313d71ff7d6 Jul 07 15:05:03.927650 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:03.927609 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1529a40-d0ce-481c-8233-200713ff5d42-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:05:03.927833 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:03.927672 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1529a40-d0ce-481c-8233-200713ff5d42-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:05:03.928214 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:03.928197 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1529a40-d0ce-481c-8233-200713ff5d42-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:05:03.930065 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:03.930037 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1529a40-d0ce-481c-8233-200713ff5d42-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7lnj4\" (UID: \"f1529a40-d0ce-481c-8233-200713ff5d42\") " pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:05:04.033487 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:04.033458 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gkfhx\"" Jul 07 15:05:04.041336 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:04.041319 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7lnj4" Jul 07 15:05:04.165507 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:04.165475 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7lnj4"] Jul 07 15:05:04.168395 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:05:04.168368 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1529a40_d0ce_481c_8233_200713ff5d42.slice/crio-50ebc6ade7c5ad9eacc2b968ed01c64481cc6a61b6ee95b690eea4479fb0d314 WatchSource:0}: Error finding container 50ebc6ade7c5ad9eacc2b968ed01c64481cc6a61b6ee95b690eea4479fb0d314: Status 404 returned error can't find the container with id 50ebc6ade7c5ad9eacc2b968ed01c64481cc6a61b6ee95b690eea4479fb0d314 Jul 07 15:05:04.554184 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:04.554100 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7lnj4" event={"ID":"f1529a40-d0ce-481c-8233-200713ff5d42","Type":"ContainerStarted","Data":"50ebc6ade7c5ad9eacc2b968ed01c64481cc6a61b6ee95b690eea4479fb0d314"} Jul 07 15:05:04.555168 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:04.555143 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dq8k2" event={"ID":"f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a","Type":"ContainerStarted","Data":"0652b2fcd703ed4f717afde1d7d729eaa426d9c0e131c996039f6313d71ff7d6"} Jul 07 15:05:05.561755 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:05.561713 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dq8k2" event={"ID":"f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a","Type":"ContainerStarted","Data":"e304f969b2901014d648c8811ea66c550d5f04c8a7aba952f9ad3ae383a11813"} Jul 07 15:05:06.569436 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:06.569392 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dq8k2" event={"ID":"f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a","Type":"ContainerStarted","Data":"9cb87a1ffd59fc423d358d39ef1ee574eabdf587d78dcfa0f0ed30f3503e4e86"} Jul 07 15:05:06.585005 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:06.584947 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dq8k2" podStartSLOduration=252.995459156 podStartE2EDuration="4m14.584927118s" totalImagesPullingTime="1.589467962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:00:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:06.583761093 +0000 UTC m=+255.400027142" watchObservedRunningTime="2026-07-07 15:05:06.584927118 +0000 UTC m=+255.401193158" Jul 07 15:05:06.782783 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:06.782752 2556 scope.go:117] "RemoveContainer" containerID="e4904cca71e3250cd6c8f3586d1da589f8507d6f3627e4e339362d0d54826021" Jul 07 15:05:08.578932 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:08.578901 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/1.log" Jul 07 15:05:08.579376 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:08.579016 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" event={"ID":"4faf5ae7-9e9a-41ac-aad2-9808d11d4c42","Type":"ContainerStarted","Data":"94bf903fe664e7f9026a2c4f8810c9e974c67666b419635b9e7f6f9be947e7c6"} Jul 07 15:05:08.579376 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:08.579338 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" Jul 07 15:05:08.580586 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:08.580563 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7lnj4" event={"ID":"f1529a40-d0ce-481c-8233-200713ff5d42","Type":"ContainerStarted","Data":"0a29db41e9a441f6607580f825aa88a9c757a99b11037bc2ffd97ef5a1b0b44b"} Jul 07 15:05:08.597940 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:08.597882 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" podStartSLOduration=23.102386587 podStartE2EDuration="26.597852961s" totalImagesPullingTime="3.495466374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:04:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:08.596960936 +0000 UTC m=+257.413226974" watchObservedRunningTime="2026-07-07 15:05:08.597852961 +0000 UTC m=+257.414119075" Jul 07 15:05:08.741573 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:08.741547 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-z4h8q" Jul 07 15:05:09.585322 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:09.585287 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7lnj4" event={"ID":"f1529a40-d0ce-481c-8233-200713ff5d42","Type":"ContainerStarted","Data":"d62336ffb52e10d5c5149707339b25f13bec59b1e7838aa39c6be482ef5ce603"} Jul 07 15:05:12.597380 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:12.597347 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7lnj4" event={"ID":"f1529a40-d0ce-481c-8233-200713ff5d42","Type":"ContainerStarted","Data":"b9bad9b82ac9b114a66c60121c17e4cf1c63139575bd230bb542ffe1c1f4a02f"} Jul 07 15:05:12.616744 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:12.616534 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7lnj4" podStartSLOduration=17.109513673 podStartE2EDuration="24.616512983s" totalImagesPullingTime="7.50699931s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:04:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:12.615930124 +0000 UTC m=+261.432196160" watchObservedRunningTime="2026-07-07 15:05:12.616512983 +0000 UTC m=+261.432779023" Jul 07 15:05:12.815028 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:12.814854 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-jzb97"] Jul 07 15:05:12.817156 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:12.817135 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-jzb97" Jul 07 15:05:12.819675 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:12.819612 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 15:05:12.820043 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:12.820022 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-wq4kx\"" Jul 07 15:05:12.821494 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:12.821478 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 15:05:12.832943 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:12.832919 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-jzb97"] Jul 07 15:05:12.855841 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:12.855815 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7884655bfd-48cr4"] Jul 07 15:05:12.857665 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:12.857651 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:12.861056 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:12.861034 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 15:05:12.861172 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:12.861085 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 15:05:12.861802 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:12.861768 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 15:05:12.862072 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:12.862052 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dwmxm\"" Jul 07 15:05:12.868098 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:12.868075 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 15:05:12.873880 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:12.873850 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7884655bfd-48cr4"] Jul 07 15:05:13.003406 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.003331 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4a1788c-8980-4bbc-a37c-387560ce6533-image-registry-private-configuration\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.003406 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.003362 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4a1788c-8980-4bbc-a37c-387560ce6533-trusted-ca\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.003406 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.003387 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4a1788c-8980-4bbc-a37c-387560ce6533-registry-certificates\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.003671 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.003455 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4a1788c-8980-4bbc-a37c-387560ce6533-bound-sa-token\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.003671 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.003552 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2sk6\" (UniqueName: \"kubernetes.io/projected/776f3b37-8cdd-4fd4-8d07-c78ced246a79-kube-api-access-g2sk6\") pod \"downloads-685bfb5f64-jzb97\" (UID: \"776f3b37-8cdd-4fd4-8d07-c78ced246a79\") " pod="openshift-console/downloads-685bfb5f64-jzb97" Jul 07 15:05:13.003671 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.003593 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4a1788c-8980-4bbc-a37c-387560ce6533-ca-trust-extracted\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.003671 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.003634 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4a1788c-8980-4bbc-a37c-387560ce6533-installation-pull-secrets\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.003671 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.003654 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4a1788c-8980-4bbc-a37c-387560ce6533-registry-tls\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.003839 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.003705 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7gbl\" (UniqueName: \"kubernetes.io/projected/e4a1788c-8980-4bbc-a37c-387560ce6533-kube-api-access-w7gbl\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.046832 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.046796 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7569cbff4b-4b2sk"] Jul 07 15:05:13.049079 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.049057 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:13.051330 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.051311 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 15:05:13.051432 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.051337 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 15:05:13.051496 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.051447 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 15:05:13.051549 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.051502 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 15:05:13.051739 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.051722 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 15:05:13.051849 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.051763 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4699p\"" Jul 07 15:05:13.061532 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.061512 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7569cbff4b-4b2sk"] Jul 07 15:05:13.104732 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.104653 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4a1788c-8980-4bbc-a37c-387560ce6533-bound-sa-token\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.104732 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.104700 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2sk6\" (UniqueName: \"kubernetes.io/projected/776f3b37-8cdd-4fd4-8d07-c78ced246a79-kube-api-access-g2sk6\") pod \"downloads-685bfb5f64-jzb97\" (UID: \"776f3b37-8cdd-4fd4-8d07-c78ced246a79\") " pod="openshift-console/downloads-685bfb5f64-jzb97" Jul 07 15:05:13.104732 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.104734 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4a1788c-8980-4bbc-a37c-387560ce6533-ca-trust-extracted\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.105018 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.104755 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4a1788c-8980-4bbc-a37c-387560ce6533-installation-pull-secrets\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.105018 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.104781 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4a1788c-8980-4bbc-a37c-387560ce6533-registry-tls\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.105018 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.104913 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7gbl\" (UniqueName: \"kubernetes.io/projected/e4a1788c-8980-4bbc-a37c-387560ce6533-kube-api-access-w7gbl\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.105018 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.104973 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4a1788c-8980-4bbc-a37c-387560ce6533-image-registry-private-configuration\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.105018 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.105009 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4a1788c-8980-4bbc-a37c-387560ce6533-trusted-ca\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.105271 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.105049 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4a1788c-8980-4bbc-a37c-387560ce6533-registry-certificates\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.105271 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.105208 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e4a1788c-8980-4bbc-a37c-387560ce6533-ca-trust-extracted\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.105956 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.105932 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e4a1788c-8980-4bbc-a37c-387560ce6533-registry-certificates\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.106086 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.106023 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e4a1788c-8980-4bbc-a37c-387560ce6533-trusted-ca\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.107341 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.107317 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e4a1788c-8980-4bbc-a37c-387560ce6533-image-registry-private-configuration\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.107447 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.107366 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e4a1788c-8980-4bbc-a37c-387560ce6533-installation-pull-secrets\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.107535 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.107518 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e4a1788c-8980-4bbc-a37c-387560ce6533-registry-tls\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.112310 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.112282 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2sk6\" (UniqueName: \"kubernetes.io/projected/776f3b37-8cdd-4fd4-8d07-c78ced246a79-kube-api-access-g2sk6\") pod \"downloads-685bfb5f64-jzb97\" (UID: \"776f3b37-8cdd-4fd4-8d07-c78ced246a79\") " pod="openshift-console/downloads-685bfb5f64-jzb97" Jul 07 15:05:13.112415 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.112285 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4a1788c-8980-4bbc-a37c-387560ce6533-bound-sa-token\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.112640 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.112609 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7gbl\" (UniqueName: \"kubernetes.io/projected/e4a1788c-8980-4bbc-a37c-387560ce6533-kube-api-access-w7gbl\") pod \"image-registry-7884655bfd-48cr4\" (UID: \"e4a1788c-8980-4bbc-a37c-387560ce6533\") " pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.128537 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.128516 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-jzb97" Jul 07 15:05:13.168394 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.168359 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.206426 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.206397 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84745d7f-a7ab-4124-b06c-69b2263b63a6-console-config\") pod \"console-7569cbff4b-4b2sk\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:13.206577 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.206432 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9x2h\" (UniqueName: \"kubernetes.io/projected/84745d7f-a7ab-4124-b06c-69b2263b63a6-kube-api-access-g9x2h\") pod \"console-7569cbff4b-4b2sk\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:13.206577 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.206463 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84745d7f-a7ab-4124-b06c-69b2263b63a6-service-ca\") pod \"console-7569cbff4b-4b2sk\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:13.206577 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.206490 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84745d7f-a7ab-4124-b06c-69b2263b63a6-oauth-serving-cert\") pod \"console-7569cbff4b-4b2sk\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:13.206577 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.206513 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84745d7f-a7ab-4124-b06c-69b2263b63a6-console-serving-cert\") pod \"console-7569cbff4b-4b2sk\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:13.206577 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.206531 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84745d7f-a7ab-4124-b06c-69b2263b63a6-console-oauth-config\") pod \"console-7569cbff4b-4b2sk\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:13.284825 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.284694 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-jzb97"] Jul 07 15:05:13.289930 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:05:13.289897 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod776f3b37_8cdd_4fd4_8d07_c78ced246a79.slice/crio-87677b6b370f1c5a691735d70d4ae280e27440109c35bdf37daee412720a24ed WatchSource:0}: Error finding container 87677b6b370f1c5a691735d70d4ae280e27440109c35bdf37daee412720a24ed: Status 404 returned error can't find the container with id 87677b6b370f1c5a691735d70d4ae280e27440109c35bdf37daee412720a24ed Jul 07 15:05:13.307396 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.307361 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84745d7f-a7ab-4124-b06c-69b2263b63a6-oauth-serving-cert\") pod \"console-7569cbff4b-4b2sk\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:13.307561 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.307413 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84745d7f-a7ab-4124-b06c-69b2263b63a6-console-serving-cert\") pod \"console-7569cbff4b-4b2sk\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:13.307561 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.307444 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84745d7f-a7ab-4124-b06c-69b2263b63a6-console-oauth-config\") pod \"console-7569cbff4b-4b2sk\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:13.307561 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.307521 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84745d7f-a7ab-4124-b06c-69b2263b63a6-console-config\") pod \"console-7569cbff4b-4b2sk\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:13.307561 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.307544 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9x2h\" (UniqueName: \"kubernetes.io/projected/84745d7f-a7ab-4124-b06c-69b2263b63a6-kube-api-access-g9x2h\") pod \"console-7569cbff4b-4b2sk\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:13.307772 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.307582 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84745d7f-a7ab-4124-b06c-69b2263b63a6-service-ca\") pod \"console-7569cbff4b-4b2sk\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:13.308243 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.308215 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84745d7f-a7ab-4124-b06c-69b2263b63a6-console-config\") pod \"console-7569cbff4b-4b2sk\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:13.308367 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.308315 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84745d7f-a7ab-4124-b06c-69b2263b63a6-service-ca\") pod \"console-7569cbff4b-4b2sk\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:13.308367 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.308333 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84745d7f-a7ab-4124-b06c-69b2263b63a6-oauth-serving-cert\") pod \"console-7569cbff4b-4b2sk\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:13.310138 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.310110 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84745d7f-a7ab-4124-b06c-69b2263b63a6-console-serving-cert\") pod \"console-7569cbff4b-4b2sk\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:13.310347 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.310327 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84745d7f-a7ab-4124-b06c-69b2263b63a6-console-oauth-config\") pod \"console-7569cbff4b-4b2sk\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:13.333562 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.333535 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9x2h\" (UniqueName: \"kubernetes.io/projected/84745d7f-a7ab-4124-b06c-69b2263b63a6-kube-api-access-g9x2h\") pod \"console-7569cbff4b-4b2sk\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:13.352716 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.352688 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7884655bfd-48cr4"] Jul 07 15:05:13.355566 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:05:13.355511 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4a1788c_8980_4bbc_a37c_387560ce6533.slice/crio-4d7a832c3150dac96cb3758bbbb1eb0676c32f20fad404e2b209cfd55c8bed9e WatchSource:0}: Error finding container 4d7a832c3150dac96cb3758bbbb1eb0676c32f20fad404e2b209cfd55c8bed9e: Status 404 returned error can't find the container with id 4d7a832c3150dac96cb3758bbbb1eb0676c32f20fad404e2b209cfd55c8bed9e Jul 07 15:05:13.357613 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.357598 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:13.490574 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.490536 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7569cbff4b-4b2sk"] Jul 07 15:05:13.493822 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:05:13.493793 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84745d7f_a7ab_4124_b06c_69b2263b63a6.slice/crio-9bfe775b0b0cf7ffd9d6400efde2681284be814db25eeb4920fa4cd7c3bedf5a WatchSource:0}: Error finding container 9bfe775b0b0cf7ffd9d6400efde2681284be814db25eeb4920fa4cd7c3bedf5a: Status 404 returned error can't find the container with id 9bfe775b0b0cf7ffd9d6400efde2681284be814db25eeb4920fa4cd7c3bedf5a Jul 07 15:05:13.602047 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.602014 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7569cbff4b-4b2sk" event={"ID":"84745d7f-a7ab-4124-b06c-69b2263b63a6","Type":"ContainerStarted","Data":"9bfe775b0b0cf7ffd9d6400efde2681284be814db25eeb4920fa4cd7c3bedf5a"} Jul 07 15:05:13.603479 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.603455 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7884655bfd-48cr4" event={"ID":"e4a1788c-8980-4bbc-a37c-387560ce6533","Type":"ContainerStarted","Data":"8c5fdf6d96e3a0d4afd054560f04b7939b702a19eedb64b9b4bac72880c8fc42"} Jul 07 15:05:13.603620 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.603499 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7884655bfd-48cr4" event={"ID":"e4a1788c-8980-4bbc-a37c-387560ce6533","Type":"ContainerStarted","Data":"4d7a832c3150dac96cb3758bbbb1eb0676c32f20fad404e2b209cfd55c8bed9e"} Jul 07 15:05:13.603620 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.603546 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:13.604589 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.604564 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-jzb97" event={"ID":"776f3b37-8cdd-4fd4-8d07-c78ced246a79","Type":"ContainerStarted","Data":"87677b6b370f1c5a691735d70d4ae280e27440109c35bdf37daee412720a24ed"} Jul 07 15:05:13.623194 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:13.623101 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7884655bfd-48cr4" podStartSLOduration=1.623083922 podStartE2EDuration="1.623083922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:05:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:13.622334681 +0000 UTC m=+262.438600723" watchObservedRunningTime="2026-07-07 15:05:13.623083922 +0000 UTC m=+262.439349940" Jul 07 15:05:14.922500 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:14.922464 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-q9n5d\" (UID: \"cc9cdf22-7b80-4c4e-941a-305d93a9ea4d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d" Jul 07 15:05:14.925073 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:14.925046 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cc9cdf22-7b80-4c4e-941a-305d93a9ea4d-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-q9n5d\" (UID: \"cc9cdf22-7b80-4c4e-941a-305d93a9ea4d\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d" Jul 07 15:05:15.189967 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:15.189882 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kllbm\"" Jul 07 15:05:15.197891 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:15.197851 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d" Jul 07 15:05:15.334490 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:15.334434 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d"] Jul 07 15:05:15.336996 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:05:15.336964 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc9cdf22_7b80_4c4e_941a_305d93a9ea4d.slice/crio-3ae0b7f6a39826594b72782e4b87082f38181f6450c846be0be6541a5a8238db WatchSource:0}: Error finding container 3ae0b7f6a39826594b72782e4b87082f38181f6450c846be0be6541a5a8238db: Status 404 returned error can't find the container with id 3ae0b7f6a39826594b72782e4b87082f38181f6450c846be0be6541a5a8238db Jul 07 15:05:15.613299 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:15.613263 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d" event={"ID":"cc9cdf22-7b80-4c4e-941a-305d93a9ea4d","Type":"ContainerStarted","Data":"3ae0b7f6a39826594b72782e4b87082f38181f6450c846be0be6541a5a8238db"} Jul 07 15:05:18.629309 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:18.629253 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7569cbff4b-4b2sk" event={"ID":"84745d7f-a7ab-4124-b06c-69b2263b63a6","Type":"ContainerStarted","Data":"2954f6a18540676ab1c4ead5a685b1f94699580ab20cbdd58c0b73ec440f1dc9"} Jul 07 15:05:18.631358 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:18.631326 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d" event={"ID":"cc9cdf22-7b80-4c4e-941a-305d93a9ea4d","Type":"ContainerStarted","Data":"8cac6473c698f8ecf9db620538e9ae017c34347bc0197a049fab241122d2952b"} Jul 07 15:05:18.647526 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:18.647293 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7569cbff4b-4b2sk" podStartSLOduration=0.891646462 podStartE2EDuration="5.647274823s" totalImagesPullingTime="4.755628361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:05:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:18.646628931 +0000 UTC m=+267.462894971" watchObservedRunningTime="2026-07-07 15:05:18.647274823 +0000 UTC m=+267.463540865" Jul 07 15:05:18.662557 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:18.662505 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-q9n5d" podStartSLOduration=32.753317644 podStartE2EDuration="35.662484391s" totalImagesPullingTime="2.909166747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:04:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:18.662452724 +0000 UTC m=+267.478718762" watchObservedRunningTime="2026-07-07 15:05:18.662484391 +0000 UTC m=+267.478750431" Jul 07 15:05:23.358604 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:23.358554 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:23.359123 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:23.358634 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:23.360276 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:23.360234 2556 patch_prober.go:28] interesting pod/console-7569cbff4b-4b2sk container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.17:8443/health\": dial tcp 10.134.0.17:8443: connect: connection refused" start-of-body= Jul 07 15:05:23.360410 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:23.360290 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7569cbff4b-4b2sk" podUID="84745d7f-a7ab-4124-b06c-69b2263b63a6" containerName="console" probeResult="failure" output="Get \"https://10.134.0.17:8443/health\": dial tcp 10.134.0.17:8443: connect: connection refused" Jul 07 15:05:28.177575 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.177543 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn"] Jul 07 15:05:28.180831 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.180813 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" Jul 07 15:05:28.183770 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.183739 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 07 15:05:28.183899 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.183841 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-z85vw\"" Jul 07 15:05:28.183978 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.183942 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 15:05:28.185012 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.184849 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 07 15:05:28.185012 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.184963 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 07 15:05:28.197526 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.197503 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-md95l"] Jul 07 15:05:28.200150 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.200130 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn"] Jul 07 15:05:28.200271 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.200256 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.209414 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.209384 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 15:05:28.210406 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.210386 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vwdqv\"" Jul 07 15:05:28.214271 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.214249 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 15:05:28.214750 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.214729 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 15:05:28.238256 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.238224 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddf9624c-3835-414e-8b3b-a3c26de812b9-sys\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.238406 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.238269 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d10f41d4-e793-447e-a01f-c04f1872686f-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-5qxsn\" (UID: \"d10f41d4-e793-447e-a01f-c04f1872686f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" Jul 07 15:05:28.238406 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.238335 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d10f41d4-e793-447e-a01f-c04f1872686f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-5qxsn\" (UID: \"d10f41d4-e793-447e-a01f-c04f1872686f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" Jul 07 15:05:28.238406 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.238377 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d10f41d4-e793-447e-a01f-c04f1872686f-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-5qxsn\" (UID: \"d10f41d4-e793-447e-a01f-c04f1872686f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" Jul 07 15:05:28.238573 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.238431 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqncc\" (UniqueName: \"kubernetes.io/projected/d10f41d4-e793-447e-a01f-c04f1872686f-kube-api-access-pqncc\") pod \"kube-state-metrics-7c9dbb8b7d-5qxsn\" (UID: \"d10f41d4-e793-447e-a01f-c04f1872686f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" Jul 07 15:05:28.238573 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.238462 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ddf9624c-3835-414e-8b3b-a3c26de812b9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.238573 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.238483 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ddf9624c-3835-414e-8b3b-a3c26de812b9-node-exporter-accelerators-collector-config\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.238714 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.238596 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ddf9624c-3835-414e-8b3b-a3c26de812b9-node-exporter-tls\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.238714 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.238627 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wblnz\" (UniqueName: \"kubernetes.io/projected/ddf9624c-3835-414e-8b3b-a3c26de812b9-kube-api-access-wblnz\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.238714 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.238659 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ddf9624c-3835-414e-8b3b-a3c26de812b9-node-exporter-wtmp\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.238824 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.238721 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d10f41d4-e793-447e-a01f-c04f1872686f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-5qxsn\" (UID: \"d10f41d4-e793-447e-a01f-c04f1872686f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" Jul 07 15:05:28.238824 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.238754 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ddf9624c-3835-414e-8b3b-a3c26de812b9-root\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.238824 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.238792 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d10f41d4-e793-447e-a01f-c04f1872686f-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-5qxsn\" (UID: \"d10f41d4-e793-447e-a01f-c04f1872686f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" Jul 07 15:05:28.238824 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.238816 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddf9624c-3835-414e-8b3b-a3c26de812b9-metrics-client-ca\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.239022 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.238841 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ddf9624c-3835-414e-8b3b-a3c26de812b9-node-exporter-textfile\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.340027 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.339993 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddf9624c-3835-414e-8b3b-a3c26de812b9-sys\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.340027 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.340037 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d10f41d4-e793-447e-a01f-c04f1872686f-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-5qxsn\" (UID: \"d10f41d4-e793-447e-a01f-c04f1872686f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" Jul 07 15:05:28.340281 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.340065 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d10f41d4-e793-447e-a01f-c04f1872686f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-5qxsn\" (UID: \"d10f41d4-e793-447e-a01f-c04f1872686f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" Jul 07 15:05:28.340281 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.340090 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d10f41d4-e793-447e-a01f-c04f1872686f-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-5qxsn\" (UID: \"d10f41d4-e793-447e-a01f-c04f1872686f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" Jul 07 15:05:28.340281 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.340116 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddf9624c-3835-414e-8b3b-a3c26de812b9-sys\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.340281 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.340124 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqncc\" (UniqueName: \"kubernetes.io/projected/d10f41d4-e793-447e-a01f-c04f1872686f-kube-api-access-pqncc\") pod \"kube-state-metrics-7c9dbb8b7d-5qxsn\" (UID: \"d10f41d4-e793-447e-a01f-c04f1872686f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" Jul 07 15:05:28.340281 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.340177 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ddf9624c-3835-414e-8b3b-a3c26de812b9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.340281 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.340204 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ddf9624c-3835-414e-8b3b-a3c26de812b9-node-exporter-accelerators-collector-config\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.340281 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.340280 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ddf9624c-3835-414e-8b3b-a3c26de812b9-node-exporter-tls\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.340629 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.340306 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wblnz\" (UniqueName: \"kubernetes.io/projected/ddf9624c-3835-414e-8b3b-a3c26de812b9-kube-api-access-wblnz\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.340629 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.340336 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ddf9624c-3835-414e-8b3b-a3c26de812b9-node-exporter-wtmp\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.340629 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.340377 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d10f41d4-e793-447e-a01f-c04f1872686f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-5qxsn\" (UID: \"d10f41d4-e793-447e-a01f-c04f1872686f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" Jul 07 15:05:28.340629 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.340401 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ddf9624c-3835-414e-8b3b-a3c26de812b9-root\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.340629 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.340432 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d10f41d4-e793-447e-a01f-c04f1872686f-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-5qxsn\" (UID: \"d10f41d4-e793-447e-a01f-c04f1872686f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" Jul 07 15:05:28.340629 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.340455 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddf9624c-3835-414e-8b3b-a3c26de812b9-metrics-client-ca\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.340629 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.340479 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ddf9624c-3835-414e-8b3b-a3c26de812b9-node-exporter-textfile\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.341005 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.340790 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ddf9624c-3835-414e-8b3b-a3c26de812b9-node-exporter-textfile\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.341005 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.340900 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d10f41d4-e793-447e-a01f-c04f1872686f-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-5qxsn\" (UID: \"d10f41d4-e793-447e-a01f-c04f1872686f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" Jul 07 15:05:28.341005 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.340924 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d10f41d4-e793-447e-a01f-c04f1872686f-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-5qxsn\" (UID: \"d10f41d4-e793-447e-a01f-c04f1872686f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" Jul 07 15:05:28.341005 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.340991 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ddf9624c-3835-414e-8b3b-a3c26de812b9-root\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.341200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.341028 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ddf9624c-3835-414e-8b3b-a3c26de812b9-node-exporter-accelerators-collector-config\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.341200 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:05:28.341134 2556 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 07 15:05:28.341296 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:05:28.341207 2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddf9624c-3835-414e-8b3b-a3c26de812b9-node-exporter-tls podName:ddf9624c-3835-414e-8b3b-a3c26de812b9 nodeName:}" failed. No retries permitted until 2026-07-07 15:05:28.841187564 +0000 UTC m=+277.657453584 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ddf9624c-3835-414e-8b3b-a3c26de812b9-node-exporter-tls") pod "node-exporter-md95l" (UID: "ddf9624c-3835-414e-8b3b-a3c26de812b9") : secret "node-exporter-tls" not found Jul 07 15:05:28.341296 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.341270 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddf9624c-3835-414e-8b3b-a3c26de812b9-metrics-client-ca\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.341296 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.341291 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ddf9624c-3835-414e-8b3b-a3c26de812b9-node-exporter-wtmp\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.341481 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.341346 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d10f41d4-e793-447e-a01f-c04f1872686f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-5qxsn\" (UID: \"d10f41d4-e793-447e-a01f-c04f1872686f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" Jul 07 15:05:28.343071 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.343025 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ddf9624c-3835-414e-8b3b-a3c26de812b9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.343204 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.343093 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d10f41d4-e793-447e-a01f-c04f1872686f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-5qxsn\" (UID: \"d10f41d4-e793-447e-a01f-c04f1872686f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" Jul 07 15:05:28.343595 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.343574 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d10f41d4-e793-447e-a01f-c04f1872686f-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-5qxsn\" (UID: \"d10f41d4-e793-447e-a01f-c04f1872686f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" Jul 07 15:05:28.350783 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.350750 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqncc\" (UniqueName: \"kubernetes.io/projected/d10f41d4-e793-447e-a01f-c04f1872686f-kube-api-access-pqncc\") pod \"kube-state-metrics-7c9dbb8b7d-5qxsn\" (UID: \"d10f41d4-e793-447e-a01f-c04f1872686f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" Jul 07 15:05:28.350924 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.350821 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wblnz\" (UniqueName: \"kubernetes.io/projected/ddf9624c-3835-414e-8b3b-a3c26de812b9-kube-api-access-wblnz\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.492051 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.491947 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" Jul 07 15:05:28.845122 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.845085 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ddf9624c-3835-414e-8b3b-a3c26de812b9-node-exporter-tls\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:28.847457 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:28.847433 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ddf9624c-3835-414e-8b3b-a3c26de812b9-node-exporter-tls\") pod \"node-exporter-md95l\" (UID: \"ddf9624c-3835-414e-8b3b-a3c26de812b9\") " pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:29.111499 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:29.111422 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-md95l" Jul 07 15:05:31.173572 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.173535 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-84d5487686-dxwqc"] Jul 07 15:05:31.176586 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.176564 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.179321 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.179298 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-9vz55\"" Jul 07 15:05:31.180752 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.180461 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 07 15:05:31.180752 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.180461 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 07 15:05:31.180752 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.180564 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-a444a27mgm3r2\"" Jul 07 15:05:31.180752 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.180471 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 07 15:05:31.180752 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.180667 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 07 15:05:31.180752 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.180748 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 07 15:05:31.190172 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.190145 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-84d5487686-dxwqc"] Jul 07 15:05:31.270969 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.270930 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/46ab3c80-fb4e-46a0-914f-a873aec066ba-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.271164 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.271005 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/46ab3c80-fb4e-46a0-914f-a873aec066ba-secret-thanos-querier-tls\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.271164 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.271038 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46ab3c80-fb4e-46a0-914f-a873aec066ba-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.271164 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.271074 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46ab3c80-fb4e-46a0-914f-a873aec066ba-metrics-client-ca\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.271353 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.271201 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/46ab3c80-fb4e-46a0-914f-a873aec066ba-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.271353 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.271261 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/46ab3c80-fb4e-46a0-914f-a873aec066ba-secret-grpc-tls\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.271353 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.271319 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46ab3c80-fb4e-46a0-914f-a873aec066ba-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.271475 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.271354 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz66n\" (UniqueName: \"kubernetes.io/projected/46ab3c80-fb4e-46a0-914f-a873aec066ba-kube-api-access-bz66n\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.371927 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.371861 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/46ab3c80-fb4e-46a0-914f-a873aec066ba-secret-thanos-querier-tls\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.372125 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.371944 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46ab3c80-fb4e-46a0-914f-a873aec066ba-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.372125 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.371988 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46ab3c80-fb4e-46a0-914f-a873aec066ba-metrics-client-ca\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.372125 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.372034 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/46ab3c80-fb4e-46a0-914f-a873aec066ba-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.372125 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.372076 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/46ab3c80-fb4e-46a0-914f-a873aec066ba-secret-grpc-tls\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.372125 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.372115 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46ab3c80-fb4e-46a0-914f-a873aec066ba-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.372384 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.372315 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz66n\" (UniqueName: \"kubernetes.io/projected/46ab3c80-fb4e-46a0-914f-a873aec066ba-kube-api-access-bz66n\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.372446 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.372379 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/46ab3c80-fb4e-46a0-914f-a873aec066ba-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.372840 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.372777 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46ab3c80-fb4e-46a0-914f-a873aec066ba-metrics-client-ca\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.375051 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.375006 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/46ab3c80-fb4e-46a0-914f-a873aec066ba-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.375051 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.375006 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/46ab3c80-fb4e-46a0-914f-a873aec066ba-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.375225 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.375190 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/46ab3c80-fb4e-46a0-914f-a873aec066ba-secret-thanos-querier-tls\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.375437 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.375413 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/46ab3c80-fb4e-46a0-914f-a873aec066ba-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.375556 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.375471 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/46ab3c80-fb4e-46a0-914f-a873aec066ba-secret-grpc-tls\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.375625 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.375609 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/46ab3c80-fb4e-46a0-914f-a873aec066ba-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.381347 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.381321 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz66n\" (UniqueName: \"kubernetes.io/projected/46ab3c80-fb4e-46a0-914f-a873aec066ba-kube-api-access-bz66n\") pod \"thanos-querier-84d5487686-dxwqc\" (UID: \"46ab3c80-fb4e-46a0-914f-a873aec066ba\") " pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:31.490842 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:31.490766 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:32.593716 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.593680 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-75b4cf9bdb-65swh"] Jul 07 15:05:32.596765 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.596746 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.601322 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.601215 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 07 15:05:32.601322 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.601297 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 15:05:32.601506 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.601297 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 07 15:05:32.601506 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.601368 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dcgfb342lgftn\"" Jul 07 15:05:32.601506 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.601302 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-6rjb5\"" Jul 07 15:05:32.601743 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.601593 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 07 15:05:32.609710 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.609686 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-75b4cf9bdb-65swh"] Jul 07 15:05:32.683927 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.683893 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f6f9475c-cfa1-424b-a7be-b5601290ecb9-metrics-server-audit-profiles\") pod \"metrics-server-75b4cf9bdb-65swh\" (UID: \"f6f9475c-cfa1-424b-a7be-b5601290ecb9\") " pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.684126 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.683949 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f6f9475c-cfa1-424b-a7be-b5601290ecb9-secret-metrics-server-tls\") pod \"metrics-server-75b4cf9bdb-65swh\" (UID: \"f6f9475c-cfa1-424b-a7be-b5601290ecb9\") " pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.684126 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.684003 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f6f9475c-cfa1-424b-a7be-b5601290ecb9-audit-log\") pod \"metrics-server-75b4cf9bdb-65swh\" (UID: \"f6f9475c-cfa1-424b-a7be-b5601290ecb9\") " pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.684126 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.684032 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f6f9475c-cfa1-424b-a7be-b5601290ecb9-client-ca-bundle\") pod \"metrics-server-75b4cf9bdb-65swh\" (UID: \"f6f9475c-cfa1-424b-a7be-b5601290ecb9\") " pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.684126 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.684102 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zdzw\" (UniqueName: \"kubernetes.io/projected/f6f9475c-cfa1-424b-a7be-b5601290ecb9-kube-api-access-9zdzw\") pod \"metrics-server-75b4cf9bdb-65swh\" (UID: \"f6f9475c-cfa1-424b-a7be-b5601290ecb9\") " pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.684334 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.684128 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6f9475c-cfa1-424b-a7be-b5601290ecb9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-75b4cf9bdb-65swh\" (UID: \"f6f9475c-cfa1-424b-a7be-b5601290ecb9\") " pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.684334 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.684189 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f6f9475c-cfa1-424b-a7be-b5601290ecb9-secret-metrics-server-client-certs\") pod \"metrics-server-75b4cf9bdb-65swh\" (UID: \"f6f9475c-cfa1-424b-a7be-b5601290ecb9\") " pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.785002 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.784958 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zdzw\" (UniqueName: \"kubernetes.io/projected/f6f9475c-cfa1-424b-a7be-b5601290ecb9-kube-api-access-9zdzw\") pod \"metrics-server-75b4cf9bdb-65swh\" (UID: \"f6f9475c-cfa1-424b-a7be-b5601290ecb9\") " pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.785002 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.785016 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6f9475c-cfa1-424b-a7be-b5601290ecb9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-75b4cf9bdb-65swh\" (UID: \"f6f9475c-cfa1-424b-a7be-b5601290ecb9\") " pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.785261 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.785051 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f6f9475c-cfa1-424b-a7be-b5601290ecb9-secret-metrics-server-client-certs\") pod \"metrics-server-75b4cf9bdb-65swh\" (UID: \"f6f9475c-cfa1-424b-a7be-b5601290ecb9\") " pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.785261 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.785252 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f6f9475c-cfa1-424b-a7be-b5601290ecb9-metrics-server-audit-profiles\") pod \"metrics-server-75b4cf9bdb-65swh\" (UID: \"f6f9475c-cfa1-424b-a7be-b5601290ecb9\") " pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.785356 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.785288 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f6f9475c-cfa1-424b-a7be-b5601290ecb9-secret-metrics-server-tls\") pod \"metrics-server-75b4cf9bdb-65swh\" (UID: \"f6f9475c-cfa1-424b-a7be-b5601290ecb9\") " pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.785356 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.785313 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f6f9475c-cfa1-424b-a7be-b5601290ecb9-audit-log\") pod \"metrics-server-75b4cf9bdb-65swh\" (UID: \"f6f9475c-cfa1-424b-a7be-b5601290ecb9\") " pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.785356 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.785345 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f6f9475c-cfa1-424b-a7be-b5601290ecb9-client-ca-bundle\") pod \"metrics-server-75b4cf9bdb-65swh\" (UID: \"f6f9475c-cfa1-424b-a7be-b5601290ecb9\") " pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.785757 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.785712 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f6f9475c-cfa1-424b-a7be-b5601290ecb9-audit-log\") pod \"metrics-server-75b4cf9bdb-65swh\" (UID: \"f6f9475c-cfa1-424b-a7be-b5601290ecb9\") " pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.785914 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.785890 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6f9475c-cfa1-424b-a7be-b5601290ecb9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-75b4cf9bdb-65swh\" (UID: \"f6f9475c-cfa1-424b-a7be-b5601290ecb9\") " pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.786353 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.786276 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f6f9475c-cfa1-424b-a7be-b5601290ecb9-metrics-server-audit-profiles\") pod \"metrics-server-75b4cf9bdb-65swh\" (UID: \"f6f9475c-cfa1-424b-a7be-b5601290ecb9\") " pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.787989 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.787959 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f6f9475c-cfa1-424b-a7be-b5601290ecb9-secret-metrics-server-client-certs\") pod \"metrics-server-75b4cf9bdb-65swh\" (UID: \"f6f9475c-cfa1-424b-a7be-b5601290ecb9\") " pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.788190 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.788146 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f6f9475c-cfa1-424b-a7be-b5601290ecb9-secret-metrics-server-tls\") pod \"metrics-server-75b4cf9bdb-65swh\" (UID: \"f6f9475c-cfa1-424b-a7be-b5601290ecb9\") " pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.788190 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.788175 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f6f9475c-cfa1-424b-a7be-b5601290ecb9-client-ca-bundle\") pod \"metrics-server-75b4cf9bdb-65swh\" (UID: \"f6f9475c-cfa1-424b-a7be-b5601290ecb9\") " pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.793268 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.793238 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zdzw\" (UniqueName: \"kubernetes.io/projected/f6f9475c-cfa1-424b-a7be-b5601290ecb9-kube-api-access-9zdzw\") pod \"metrics-server-75b4cf9bdb-65swh\" (UID: \"f6f9475c-cfa1-424b-a7be-b5601290ecb9\") " pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.909433 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:32.909353 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:32.972802 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:05:32.972768 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddf9624c_3835_414e_8b3b_a3c26de812b9.slice/crio-a5cb9ce5562d0a835fc5d8a184e347ac27f52ad274c8263bc98a37256da5ba36 WatchSource:0}: Error finding container a5cb9ce5562d0a835fc5d8a184e347ac27f52ad274c8263bc98a37256da5ba36: Status 404 returned error can't find the container with id a5cb9ce5562d0a835fc5d8a184e347ac27f52ad274c8263bc98a37256da5ba36 Jul 07 15:05:33.042737 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.041891 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7569cbff4b-4b2sk"] Jul 07 15:05:33.073381 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.073352 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6cd7b76fbd-ttm4h"] Jul 07 15:05:33.081444 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.081357 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.085970 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.085734 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cd7b76fbd-ttm4h"] Jul 07 15:05:33.089514 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.089493 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 15:05:33.127073 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.126856 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-75b4cf9bdb-65swh"] Jul 07 15:05:33.129953 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:05:33.129925 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6f9475c_cfa1_424b_a7be_b5601290ecb9.slice/crio-8171e461c3851e550977b630e495b6ede819a8758472025ae73d0473aec53dc7 WatchSource:0}: Error finding container 8171e461c3851e550977b630e495b6ede819a8758472025ae73d0473aec53dc7: Status 404 returned error can't find the container with id 8171e461c3851e550977b630e495b6ede819a8758472025ae73d0473aec53dc7 Jul 07 15:05:33.173184 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.173154 2556 patch_prober.go:28] interesting pod/image-registry-7884655bfd-48cr4 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 07 15:05:33.173315 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.173214 2556 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7884655bfd-48cr4" podUID="e4a1788c-8980-4bbc-a37c-387560ce6533" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 15:05:33.190283 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.190248 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-oauth-serving-cert\") pod \"console-6cd7b76fbd-ttm4h\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.190424 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.190291 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5ba963a-f45b-412e-abbd-ba8c79f6324e-console-serving-cert\") pod \"console-6cd7b76fbd-ttm4h\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.190424 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.190315 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5ba963a-f45b-412e-abbd-ba8c79f6324e-console-oauth-config\") pod \"console-6cd7b76fbd-ttm4h\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.190424 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.190334 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-trusted-ca-bundle\") pod \"console-6cd7b76fbd-ttm4h\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.190424 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.190388 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbbrh\" (UniqueName: \"kubernetes.io/projected/f5ba963a-f45b-412e-abbd-ba8c79f6324e-kube-api-access-bbbrh\") pod \"console-6cd7b76fbd-ttm4h\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.190424 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.190420 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-service-ca\") pod \"console-6cd7b76fbd-ttm4h\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.190575 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.190483 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-console-config\") pod \"console-6cd7b76fbd-ttm4h\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.291214 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.291174 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-console-config\") pod \"console-6cd7b76fbd-ttm4h\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.291370 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.291221 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-oauth-serving-cert\") pod \"console-6cd7b76fbd-ttm4h\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.291370 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.291245 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5ba963a-f45b-412e-abbd-ba8c79f6324e-console-serving-cert\") pod \"console-6cd7b76fbd-ttm4h\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.291370 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.291269 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5ba963a-f45b-412e-abbd-ba8c79f6324e-console-oauth-config\") pod \"console-6cd7b76fbd-ttm4h\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.291370 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.291290 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-trusted-ca-bundle\") pod \"console-6cd7b76fbd-ttm4h\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.291370 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.291313 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbbrh\" (UniqueName: \"kubernetes.io/projected/f5ba963a-f45b-412e-abbd-ba8c79f6324e-kube-api-access-bbbrh\") pod \"console-6cd7b76fbd-ttm4h\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.291370 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.291344 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-service-ca\") pod \"console-6cd7b76fbd-ttm4h\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.292085 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.292058 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-console-config\") pod \"console-6cd7b76fbd-ttm4h\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.292223 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.292058 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-oauth-serving-cert\") pod \"console-6cd7b76fbd-ttm4h\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.292613 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.292580 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-trusted-ca-bundle\") pod \"console-6cd7b76fbd-ttm4h\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.292968 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.292941 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-service-ca\") pod \"console-6cd7b76fbd-ttm4h\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.293899 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.293842 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5ba963a-f45b-412e-abbd-ba8c79f6324e-console-serving-cert\") pod \"console-6cd7b76fbd-ttm4h\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.293991 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.293937 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5ba963a-f45b-412e-abbd-ba8c79f6324e-console-oauth-config\") pod \"console-6cd7b76fbd-ttm4h\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.300272 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.300250 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbbrh\" (UniqueName: \"kubernetes.io/projected/f5ba963a-f45b-412e-abbd-ba8c79f6324e-kube-api-access-bbbrh\") pod \"console-6cd7b76fbd-ttm4h\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.300551 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:05:33.300527 2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-l7998" podUID="1266e9ff-4ff3-4177-8402-1550a338121a" Jul 07 15:05:33.355752 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.355719 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn"] Jul 07 15:05:33.358404 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:05:33.358371 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd10f41d4_e793_447e_a01f_c04f1872686f.slice/crio-f588ae42231d824c59a23990016246ae9d9e5531d009f23e8ece118ea533d77d WatchSource:0}: Error finding container f588ae42231d824c59a23990016246ae9d9e5531d009f23e8ece118ea533d77d: Status 404 returned error can't find the container with id f588ae42231d824c59a23990016246ae9d9e5531d009f23e8ece118ea533d77d Jul 07 15:05:33.358404 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.358408 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-84d5487686-dxwqc"] Jul 07 15:05:33.362405 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:05:33.362377 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46ab3c80_fb4e_46a0_914f_a873aec066ba.slice/crio-9a726d0669b5c0a47fe0219f85434dc47cbdd9af62009dae8b0215be0e4128a6 WatchSource:0}: Error finding container 9a726d0669b5c0a47fe0219f85434dc47cbdd9af62009dae8b0215be0e4128a6: Status 404 returned error can't find the container with id 9a726d0669b5c0a47fe0219f85434dc47cbdd9af62009dae8b0215be0e4128a6 Jul 07 15:05:33.396781 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.396748 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:33.557033 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.556995 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6cd7b76fbd-ttm4h"] Jul 07 15:05:33.560253 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:05:33.560212 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5ba963a_f45b_412e_abbd_ba8c79f6324e.slice/crio-275db41af8fbb1e23c57b60ab4053b73cca67a8384fc1bfc8f3a0508573cb8e8 WatchSource:0}: Error finding container 275db41af8fbb1e23c57b60ab4053b73cca67a8384fc1bfc8f3a0508573cb8e8: Status 404 returned error can't find the container with id 275db41af8fbb1e23c57b60ab4053b73cca67a8384fc1bfc8f3a0508573cb8e8 Jul 07 15:05:33.694648 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.693602 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd7b76fbd-ttm4h" event={"ID":"f5ba963a-f45b-412e-abbd-ba8c79f6324e","Type":"ContainerStarted","Data":"427660e54bf0a97fbc88a089143614f85f03a29662f376829f8710cb4be84875"} Jul 07 15:05:33.694648 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.693661 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd7b76fbd-ttm4h" event={"ID":"f5ba963a-f45b-412e-abbd-ba8c79f6324e","Type":"ContainerStarted","Data":"275db41af8fbb1e23c57b60ab4053b73cca67a8384fc1bfc8f3a0508573cb8e8"} Jul 07 15:05:33.698747 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.696922 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-jzb97" event={"ID":"776f3b37-8cdd-4fd4-8d07-c78ced246a79","Type":"ContainerStarted","Data":"7cf21dd6a07985311787d53f229b5936b1241c687538617a02a0cc73eb84147f"} Jul 07 15:05:33.698747 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.697609 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-jzb97" Jul 07 15:05:33.701387 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.700393 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" event={"ID":"d10f41d4-e793-447e-a01f-c04f1872686f","Type":"ContainerStarted","Data":"f588ae42231d824c59a23990016246ae9d9e5531d009f23e8ece118ea533d77d"} Jul 07 15:05:33.702500 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.702453 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" event={"ID":"f6f9475c-cfa1-424b-a7be-b5601290ecb9","Type":"ContainerStarted","Data":"8171e461c3851e550977b630e495b6ede819a8758472025ae73d0473aec53dc7"} Jul 07 15:05:33.704685 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.704592 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-md95l" event={"ID":"ddf9624c-3835-414e-8b3b-a3c26de812b9","Type":"ContainerStarted","Data":"a5cb9ce5562d0a835fc5d8a184e347ac27f52ad274c8263bc98a37256da5ba36"} Jul 07 15:05:33.706329 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.706274 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l7998" Jul 07 15:05:33.706890 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.706577 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" event={"ID":"46ab3c80-fb4e-46a0-914f-a873aec066ba","Type":"ContainerStarted","Data":"9a726d0669b5c0a47fe0219f85434dc47cbdd9af62009dae8b0215be0e4128a6"} Jul 07 15:05:33.712782 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.712719 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-jzb97" Jul 07 15:05:33.713996 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.713658 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6cd7b76fbd-ttm4h" podStartSLOduration=0.713640072 podStartE2EDuration="713.640072ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:05:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:33.712969956 +0000 UTC m=+282.529235996" watchObservedRunningTime="2026-07-07 15:05:33.713640072 +0000 UTC m=+282.529906113" Jul 07 15:05:33.732903 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:33.732755 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-jzb97" podStartSLOduration=1.866589592 podStartE2EDuration="21.732736555s" totalImagesPullingTime="19.866146963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:05:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:33.730728031 +0000 UTC m=+282.546994072" watchObservedRunningTime="2026-07-07 15:05:33.732736555 +0000 UTC m=+282.549002595" Jul 07 15:05:34.400048 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.399308 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 15:05:34.450975 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.449377 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 15:05:34.450975 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.449559 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.452602 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.452237 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 15:05:34.452602 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.452598 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 15:05:34.455379 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.453038 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 15:05:34.455379 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.454624 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4a6l4ogcg9dp2\"" Jul 07 15:05:34.455379 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.454844 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 15:05:34.455379 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.455108 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 15:05:34.455379 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.455273 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 15:05:34.455695 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.455426 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 15:05:34.455695 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.455647 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 15:05:34.458146 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.455829 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-278cd\"" Jul 07 15:05:34.458146 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.456017 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 15:05:34.458146 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.456073 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 15:05:34.458146 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.456016 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 15:05:34.458146 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.456157 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 15:05:34.458146 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.456264 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 15:05:34.458146 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.457960 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 15:05:34.506182 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.505321 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ca68156-8bab-4e82-85c4-0695c94f3f64-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.506182 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.505367 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ca68156-8bab-4e82-85c4-0695c94f3f64-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.506182 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.505393 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ca68156-8bab-4e82-85c4-0695c94f3f64-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.506182 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.505415 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-config\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.506182 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.505446 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.506182 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.505483 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.506182 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.505513 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.506182 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.505542 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7ca68156-8bab-4e82-85c4-0695c94f3f64-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.506182 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.505566 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pqlt\" (UniqueName: \"kubernetes.io/projected/7ca68156-8bab-4e82-85c4-0695c94f3f64-kube-api-access-2pqlt\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.506182 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.505594 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-web-config\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.506182 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.505627 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ca68156-8bab-4e82-85c4-0695c94f3f64-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.506182 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.505661 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7ca68156-8bab-4e82-85c4-0695c94f3f64-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.506182 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.505696 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.506941 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.505724 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.506941 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.505752 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.506941 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.505777 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.506941 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.505824 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ca68156-8bab-4e82-85c4-0695c94f3f64-config-out\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.506941 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.505850 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7ca68156-8bab-4e82-85c4-0695c94f3f64-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.506941 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.505903 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7ca68156-8bab-4e82-85c4-0695c94f3f64-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.506941 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.505956 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ca68156-8bab-4e82-85c4-0695c94f3f64-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.606760 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.606724 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ca68156-8bab-4e82-85c4-0695c94f3f64-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.606962 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.606792 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ca68156-8bab-4e82-85c4-0695c94f3f64-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.606962 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.606829 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ca68156-8bab-4e82-85c4-0695c94f3f64-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.606962 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.606862 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ca68156-8bab-4e82-85c4-0695c94f3f64-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.606962 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.606920 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-config\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.606962 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.606952 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.607245 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.606998 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.607245 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.607032 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.607245 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.607060 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7ca68156-8bab-4e82-85c4-0695c94f3f64-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.607245 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.607085 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pqlt\" (UniqueName: \"kubernetes.io/projected/7ca68156-8bab-4e82-85c4-0695c94f3f64-kube-api-access-2pqlt\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.607245 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.607120 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-web-config\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.607245 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.607149 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ca68156-8bab-4e82-85c4-0695c94f3f64-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.607245 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.607170 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7ca68156-8bab-4e82-85c4-0695c94f3f64-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.607245 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.607212 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.607245 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.607241 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.607673 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.607269 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.607673 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.607292 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.607673 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.607341 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ca68156-8bab-4e82-85c4-0695c94f3f64-config-out\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.607673 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.607367 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7ca68156-8bab-4e82-85c4-0695c94f3f64-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.607673 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.607396 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7ca68156-8bab-4e82-85c4-0695c94f3f64-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.607673 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.607608 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7ca68156-8bab-4e82-85c4-0695c94f3f64-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.609425 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.609394 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ca68156-8bab-4e82-85c4-0695c94f3f64-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.611499 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.611168 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ca68156-8bab-4e82-85c4-0695c94f3f64-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.612123 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.612099 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-web-config\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.612227 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.612102 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7ca68156-8bab-4e82-85c4-0695c94f3f64-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.612618 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.612593 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7ca68156-8bab-4e82-85c4-0695c94f3f64-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.613276 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.613252 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7ca68156-8bab-4e82-85c4-0695c94f3f64-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.615308 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.615284 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.615411 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.615286 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7ca68156-8bab-4e82-85c4-0695c94f3f64-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.616246 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.616220 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7ca68156-8bab-4e82-85c4-0695c94f3f64-config-out\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.616378 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.616312 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.616556 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.616533 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ca68156-8bab-4e82-85c4-0695c94f3f64-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.619952 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.618945 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pqlt\" (UniqueName: \"kubernetes.io/projected/7ca68156-8bab-4e82-85c4-0695c94f3f64-kube-api-access-2pqlt\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.620111 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.620059 2556 patch_prober.go:28] interesting pod/image-registry-7884655bfd-48cr4 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 07 15:05:34.620212 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.620183 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7884655bfd-48cr4" podUID="e4a1788c-8980-4bbc-a37c-387560ce6533" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 15:05:34.620737 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.620674 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.621460 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.621420 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7ca68156-8bab-4e82-85c4-0695c94f3f64-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.622716 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.622680 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.623477 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.623436 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.623561 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.623447 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-config\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.624792 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.624415 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.625426 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.625159 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7ca68156-8bab-4e82-85c4-0695c94f3f64-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7ca68156-8bab-4e82-85c4-0695c94f3f64\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:34.766819 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:34.766720 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:36.411603 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:36.411120 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 15:05:36.724360 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:36.724157 2556 generic.go:356] "Generic (PLEG): container finished" podID="ddf9624c-3835-414e-8b3b-a3c26de812b9" containerID="7139acd695c0c87a4e857ade2de031fb6b5f803e9377f4fe5d641fd01371492e" exitCode=0 Jul 07 15:05:36.724360 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:36.724268 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-md95l" event={"ID":"ddf9624c-3835-414e-8b3b-a3c26de812b9","Type":"ContainerDied","Data":"7139acd695c0c87a4e857ade2de031fb6b5f803e9377f4fe5d641fd01371492e"} Jul 07 15:05:36.730346 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:36.730214 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" event={"ID":"d10f41d4-e793-447e-a01f-c04f1872686f","Type":"ContainerStarted","Data":"b0ad2c202e181344b445e337f169b9ced34e54dff2998bab32b9451401c25a9a"} Jul 07 15:05:36.730346 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:36.730277 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" event={"ID":"d10f41d4-e793-447e-a01f-c04f1872686f","Type":"ContainerStarted","Data":"6372b2621ff702e4f28187f697e91fba8b480e40ba80797810da5f1db6c99e61"} Jul 07 15:05:36.730346 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:36.730305 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" event={"ID":"d10f41d4-e793-447e-a01f-c04f1872686f","Type":"ContainerStarted","Data":"ebb7b60a0a99f7f310bc3130acf360e4d822b2604648c0daa3b7c2a936284c92"} Jul 07 15:05:36.732715 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:36.732654 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" event={"ID":"f6f9475c-cfa1-424b-a7be-b5601290ecb9","Type":"ContainerStarted","Data":"f84a01c5b7737a41ff81e7e9909ef3f96eeede4ae5f5b9b93ec6f6272bda4c48"} Jul 07 15:05:36.734312 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:36.734293 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ca68156-8bab-4e82-85c4-0695c94f3f64","Type":"ContainerStarted","Data":"9e0ff06360ac314e2b4b0aaed84b5d58c2015600afc72bae3dd0625f2b1c93de"} Jul 07 15:05:36.767320 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:36.765692 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-5qxsn" podStartSLOduration=5.9057911579999995 podStartE2EDuration="8.765671573s" totalImagesPullingTime="2.859880415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:05:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:36.764344575 +0000 UTC m=+285.580610616" watchObservedRunningTime="2026-07-07 15:05:36.765671573 +0000 UTC m=+285.581937613" Jul 07 15:05:36.792759 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:36.792702 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" podStartSLOduration=1.708937645 podStartE2EDuration="4.792680428s" totalImagesPullingTime="3.083742783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:05:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:36.792148706 +0000 UTC m=+285.608414748" watchObservedRunningTime="2026-07-07 15:05:36.792680428 +0000 UTC m=+285.608946470" Jul 07 15:05:36.932629 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:36.932538 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls\") pod \"dns-default-l7998\" (UID: \"1266e9ff-4ff3-4177-8402-1550a338121a\") " pod="openshift-dns/dns-default-l7998" Jul 07 15:05:36.936644 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:36.936617 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1266e9ff-4ff3-4177-8402-1550a338121a-metrics-tls\") pod \"dns-default-l7998\" (UID: \"1266e9ff-4ff3-4177-8402-1550a338121a\") " pod="openshift-dns/dns-default-l7998" Jul 07 15:05:37.009373 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:37.009341 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bp52v\"" Jul 07 15:05:37.018336 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:37.018307 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l7998" Jul 07 15:05:37.597955 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:37.597888 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l7998"] Jul 07 15:05:37.741793 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:37.741712 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-md95l" event={"ID":"ddf9624c-3835-414e-8b3b-a3c26de812b9","Type":"ContainerStarted","Data":"b8eca2e9542b1659f26bd24fea7a1a792bf8b9e3aacf0cbf658077cc922fb88f"} Jul 07 15:05:37.759237 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:05:37.759207 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1266e9ff_4ff3_4177_8402_1550a338121a.slice/crio-1c36549f066e384a6cccdb75539eb147c2e912807f60c5e466d2a8b31737210d WatchSource:0}: Error finding container 1c36549f066e384a6cccdb75539eb147c2e912807f60c5e466d2a8b31737210d: Status 404 returned error can't find the container with id 1c36549f066e384a6cccdb75539eb147c2e912807f60c5e466d2a8b31737210d Jul 07 15:05:38.748821 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:38.748780 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-md95l" event={"ID":"ddf9624c-3835-414e-8b3b-a3c26de812b9","Type":"ContainerStarted","Data":"a08453733d83dada857aca208c5c5ce0303245630a927c90700cd7b25d26cf10"} Jul 07 15:05:38.751666 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:38.751630 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" event={"ID":"46ab3c80-fb4e-46a0-914f-a873aec066ba","Type":"ContainerStarted","Data":"e83cc6435089f0079d4555d662a19ed439c5f83f77554f1326370b2060a769c7"} Jul 07 15:05:38.751814 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:38.751713 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" event={"ID":"46ab3c80-fb4e-46a0-914f-a873aec066ba","Type":"ContainerStarted","Data":"71f4ab0525d74d1ac28e07de1f08d59d24ee7ef48b5236c9b1ae485253786eb3"} Jul 07 15:05:38.751814 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:38.751751 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" event={"ID":"46ab3c80-fb4e-46a0-914f-a873aec066ba","Type":"ContainerStarted","Data":"9050c464c6dbde6259fa0ec67fb1c6ae9ab1620fd9ce944403dc58541b4206ab"} Jul 07 15:05:38.753108 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:38.753079 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l7998" event={"ID":"1266e9ff-4ff3-4177-8402-1550a338121a","Type":"ContainerStarted","Data":"1c36549f066e384a6cccdb75539eb147c2e912807f60c5e466d2a8b31737210d"} Jul 07 15:05:38.768095 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:38.768050 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-md95l" podStartSLOduration=7.369576129 podStartE2EDuration="10.768035567s" totalImagesPullingTime="3.242334349s" totalInitContainerRuntime="156.125089ms" isStatefulPod=true podCreationTimestamp="2026-07-07 15:05:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:38.766107532 +0000 UTC m=+287.582373572" watchObservedRunningTime="2026-07-07 15:05:38.768035567 +0000 UTC m=+287.584301605" Jul 07 15:05:39.760340 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:39.760257 2556 generic.go:356] "Generic (PLEG): container finished" podID="7ca68156-8bab-4e82-85c4-0695c94f3f64" containerID="2ac48890e6809dbe032af89ade1bd3281939c09c623ac202997092bf2cad8172" exitCode=0 Jul 07 15:05:39.760938 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:39.760361 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ca68156-8bab-4e82-85c4-0695c94f3f64","Type":"ContainerDied","Data":"2ac48890e6809dbe032af89ade1bd3281939c09c623ac202997092bf2cad8172"} Jul 07 15:05:40.769301 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:40.769266 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" event={"ID":"46ab3c80-fb4e-46a0-914f-a873aec066ba","Type":"ContainerStarted","Data":"011bcc63a61bfc88ddc630de7be6405dc6f53e34652e07acdc0d26d649c45ac7"} Jul 07 15:05:40.769702 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:40.769347 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" event={"ID":"46ab3c80-fb4e-46a0-914f-a873aec066ba","Type":"ContainerStarted","Data":"bf9bdcc3276602e83ce31b17c258c95db2c2490748038391f49f1a49b6096319"} Jul 07 15:05:41.780394 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:41.780356 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l7998" event={"ID":"1266e9ff-4ff3-4177-8402-1550a338121a","Type":"ContainerStarted","Data":"f078828a00fe9dcbbc4bfb4aec50899d34576ed63482f4e0d391bdf900af6775"} Jul 07 15:05:41.780961 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:41.780411 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l7998" event={"ID":"1266e9ff-4ff3-4177-8402-1550a338121a","Type":"ContainerStarted","Data":"0d5ed7098ef131341004a44b40a56596db0cda63292b28b40fb2b24d0f2ff698"} Jul 07 15:05:41.781156 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:41.781136 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-l7998" Jul 07 15:05:41.794355 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:41.794026 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" event={"ID":"46ab3c80-fb4e-46a0-914f-a873aec066ba","Type":"ContainerStarted","Data":"f0e83d017ae4d1e9a3a677cd8188a647573924f0a171fc174d76e1f1344167dd"} Jul 07 15:05:41.794355 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:41.794121 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:41.802535 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:41.802479 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-l7998" podStartSLOduration=252.617477469 podStartE2EDuration="4m15.802457204s" totalImagesPullingTime="3.184979735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:01:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:41.797837157 +0000 UTC m=+290.614103197" watchObservedRunningTime="2026-07-07 15:05:41.802457204 +0000 UTC m=+290.618723245" Jul 07 15:05:41.825588 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:41.825465 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" podStartSLOduration=4.762465367 podStartE2EDuration="10.825444115s" totalImagesPullingTime="6.062978748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:05:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:41.822963967 +0000 UTC m=+290.639230008" watchObservedRunningTime="2026-07-07 15:05:41.825444115 +0000 UTC m=+290.641710155" Jul 07 15:05:43.173099 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:43.173061 2556 patch_prober.go:28] interesting pod/image-registry-7884655bfd-48cr4 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 07 15:05:43.173560 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:43.173118 2556 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7884655bfd-48cr4" podUID="e4a1788c-8980-4bbc-a37c-387560ce6533" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 15:05:43.397957 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:43.397915 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:43.399251 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:43.398919 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:43.400810 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:43.400457 2556 patch_prober.go:28] interesting pod/console-6cd7b76fbd-ttm4h container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.21:8443/health\": dial tcp 10.134.0.21:8443: connect: connection refused" start-of-body= Jul 07 15:05:43.400810 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:43.400530 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6cd7b76fbd-ttm4h" podUID="f5ba963a-f45b-412e-abbd-ba8c79f6324e" containerName="console" probeResult="failure" output="Get \"https://10.134.0.21:8443/health\": dial tcp 10.134.0.21:8443: connect: connection refused" Jul 07 15:05:44.613280 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:44.613252 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7884655bfd-48cr4" Jul 07 15:05:45.810712 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:45.810617 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ca68156-8bab-4e82-85c4-0695c94f3f64","Type":"ContainerStarted","Data":"6765fd314a1d5bf4bd0ddf9c133917a600eed9f99757680e852069d81cd29a79"} Jul 07 15:05:45.810712 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:45.810693 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ca68156-8bab-4e82-85c4-0695c94f3f64","Type":"ContainerStarted","Data":"9127acbbd8be30f6b0b3546f325b196a29f3232a54ba96e30ed14e438a89de55"} Jul 07 15:05:45.811214 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:45.810722 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ca68156-8bab-4e82-85c4-0695c94f3f64","Type":"ContainerStarted","Data":"33d84b17db9154e03fc154815c8d5852f52fb76a00e8e27240971d1cec8ec38a"} Jul 07 15:05:45.811214 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:45.810746 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ca68156-8bab-4e82-85c4-0695c94f3f64","Type":"ContainerStarted","Data":"4709c24d6727932ac84dfdb9d05cab4b051212725ca8d23efcbdaa2141978b09"} Jul 07 15:05:45.811214 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:45.810781 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ca68156-8bab-4e82-85c4-0695c94f3f64","Type":"ContainerStarted","Data":"b84cc7978710d6e7b36850e54c52a51579ca1cad42fbdb6dcb21297d3e0f6424"} Jul 07 15:05:45.811214 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:45.810806 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7ca68156-8bab-4e82-85c4-0695c94f3f64","Type":"ContainerStarted","Data":"2809a37cc1da41b9c9df3660deb6f1dc461fff1feaaf87e0b081f1189db0d333"} Jul 07 15:05:45.840241 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:45.840169 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.342724902 podStartE2EDuration="11.840145079s" totalImagesPullingTime="7.466680199s" totalInitContainerRuntime="30.739978ms" isStatefulPod=false podCreationTimestamp="2026-07-07 15:05:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:05:45.837848483 +0000 UTC m=+294.654114523" watchObservedRunningTime="2026-07-07 15:05:45.840145079 +0000 UTC m=+294.656411118" Jul 07 15:05:47.801415 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:47.801384 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-84d5487686-dxwqc" Jul 07 15:05:49.767397 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:49.767361 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:05:51.581639 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:51.581606 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/1.log" Jul 07 15:05:51.582100 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:51.581810 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/1.log" Jul 07 15:05:51.591948 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:51.591926 2556 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 15:05:52.802543 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:52.802510 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-l7998" Jul 07 15:05:52.910061 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:52.910027 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:52.910241 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:52.910088 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:05:53.401462 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:53.401427 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:53.405563 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:53.405537 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:05:58.073162 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.073100 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7569cbff4b-4b2sk" podUID="84745d7f-a7ab-4124-b06c-69b2263b63a6" containerName="console" containerID="cri-o://2954f6a18540676ab1c4ead5a685b1f94699580ab20cbdd58c0b73ec440f1dc9" gracePeriod=15 Jul 07 15:05:58.356605 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.356574 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:58.540027 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.539990 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84745d7f-a7ab-4124-b06c-69b2263b63a6-console-serving-cert\") pod \"84745d7f-a7ab-4124-b06c-69b2263b63a6\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " Jul 07 15:05:58.540227 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.540045 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84745d7f-a7ab-4124-b06c-69b2263b63a6-console-oauth-config\") pod \"84745d7f-a7ab-4124-b06c-69b2263b63a6\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " Jul 07 15:05:58.540227 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.540062 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84745d7f-a7ab-4124-b06c-69b2263b63a6-oauth-serving-cert\") pod \"84745d7f-a7ab-4124-b06c-69b2263b63a6\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " Jul 07 15:05:58.540227 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.540097 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g9x2h\" (UniqueName: \"kubernetes.io/projected/84745d7f-a7ab-4124-b06c-69b2263b63a6-kube-api-access-g9x2h\") pod \"84745d7f-a7ab-4124-b06c-69b2263b63a6\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " Jul 07 15:05:58.540227 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.540115 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84745d7f-a7ab-4124-b06c-69b2263b63a6-service-ca\") pod \"84745d7f-a7ab-4124-b06c-69b2263b63a6\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " Jul 07 15:05:58.540227 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.540159 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84745d7f-a7ab-4124-b06c-69b2263b63a6-console-config\") pod \"84745d7f-a7ab-4124-b06c-69b2263b63a6\" (UID: \"84745d7f-a7ab-4124-b06c-69b2263b63a6\") " Jul 07 15:05:58.540580 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.540548 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84745d7f-a7ab-4124-b06c-69b2263b63a6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "84745d7f-a7ab-4124-b06c-69b2263b63a6" (UID: "84745d7f-a7ab-4124-b06c-69b2263b63a6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:05:58.540710 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.540600 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84745d7f-a7ab-4124-b06c-69b2263b63a6-console-config" (OuterVolumeSpecName: "console-config") pod "84745d7f-a7ab-4124-b06c-69b2263b63a6" (UID: "84745d7f-a7ab-4124-b06c-69b2263b63a6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:05:58.540710 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.540610 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84745d7f-a7ab-4124-b06c-69b2263b63a6-service-ca" (OuterVolumeSpecName: "service-ca") pod "84745d7f-a7ab-4124-b06c-69b2263b63a6" (UID: "84745d7f-a7ab-4124-b06c-69b2263b63a6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:05:58.542332 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.542308 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84745d7f-a7ab-4124-b06c-69b2263b63a6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "84745d7f-a7ab-4124-b06c-69b2263b63a6" (UID: "84745d7f-a7ab-4124-b06c-69b2263b63a6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:05:58.542712 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.542679 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84745d7f-a7ab-4124-b06c-69b2263b63a6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "84745d7f-a7ab-4124-b06c-69b2263b63a6" (UID: "84745d7f-a7ab-4124-b06c-69b2263b63a6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:05:58.542712 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.542690 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84745d7f-a7ab-4124-b06c-69b2263b63a6-kube-api-access-g9x2h" (OuterVolumeSpecName: "kube-api-access-g9x2h") pod "84745d7f-a7ab-4124-b06c-69b2263b63a6" (UID: "84745d7f-a7ab-4124-b06c-69b2263b63a6"). InnerVolumeSpecName "kube-api-access-g9x2h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:05:58.640888 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.640784 2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84745d7f-a7ab-4124-b06c-69b2263b63a6-console-config\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:05:58.640888 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.640814 2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84745d7f-a7ab-4124-b06c-69b2263b63a6-console-serving-cert\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:05:58.640888 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.640826 2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84745d7f-a7ab-4124-b06c-69b2263b63a6-console-oauth-config\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:05:58.640888 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.640834 2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84745d7f-a7ab-4124-b06c-69b2263b63a6-oauth-serving-cert\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:05:58.640888 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.640842 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9x2h\" (UniqueName: \"kubernetes.io/projected/84745d7f-a7ab-4124-b06c-69b2263b63a6-kube-api-access-g9x2h\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:05:58.640888 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.640850 2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84745d7f-a7ab-4124-b06c-69b2263b63a6-service-ca\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:05:58.865879 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.865831 2556 generic.go:356] "Generic (PLEG): container finished" podID="84745d7f-a7ab-4124-b06c-69b2263b63a6" containerID="2954f6a18540676ab1c4ead5a685b1f94699580ab20cbdd58c0b73ec440f1dc9" exitCode=0 Jul 07 15:05:58.866079 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.865892 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7569cbff4b-4b2sk" Jul 07 15:05:58.866079 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.865924 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7569cbff4b-4b2sk" event={"ID":"84745d7f-a7ab-4124-b06c-69b2263b63a6","Type":"ContainerDied","Data":"2954f6a18540676ab1c4ead5a685b1f94699580ab20cbdd58c0b73ec440f1dc9"} Jul 07 15:05:58.866079 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.865978 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7569cbff4b-4b2sk" event={"ID":"84745d7f-a7ab-4124-b06c-69b2263b63a6","Type":"ContainerDied","Data":"9bfe775b0b0cf7ffd9d6400efde2681284be814db25eeb4920fa4cd7c3bedf5a"} Jul 07 15:05:58.866079 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.865999 2556 scope.go:117] "RemoveContainer" containerID="2954f6a18540676ab1c4ead5a685b1f94699580ab20cbdd58c0b73ec440f1dc9" Jul 07 15:05:58.875039 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.875017 2556 scope.go:117] "RemoveContainer" containerID="2954f6a18540676ab1c4ead5a685b1f94699580ab20cbdd58c0b73ec440f1dc9" Jul 07 15:05:58.875334 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:05:58.875313 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2954f6a18540676ab1c4ead5a685b1f94699580ab20cbdd58c0b73ec440f1dc9\": container with ID starting with 2954f6a18540676ab1c4ead5a685b1f94699580ab20cbdd58c0b73ec440f1dc9 not found: ID does not exist" containerID="2954f6a18540676ab1c4ead5a685b1f94699580ab20cbdd58c0b73ec440f1dc9" Jul 07 15:05:58.875383 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.875344 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2954f6a18540676ab1c4ead5a685b1f94699580ab20cbdd58c0b73ec440f1dc9"} err="failed to get container status \"2954f6a18540676ab1c4ead5a685b1f94699580ab20cbdd58c0b73ec440f1dc9\": rpc error: code = NotFound desc = could not find container \"2954f6a18540676ab1c4ead5a685b1f94699580ab20cbdd58c0b73ec440f1dc9\": container with ID starting with 2954f6a18540676ab1c4ead5a685b1f94699580ab20cbdd58c0b73ec440f1dc9 not found: ID does not exist" Jul 07 15:05:58.886595 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.886570 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7569cbff4b-4b2sk"] Jul 07 15:05:58.890405 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:58.890381 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7569cbff4b-4b2sk"] Jul 07 15:05:59.788200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:05:59.788169 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84745d7f-a7ab-4124-b06c-69b2263b63a6" path="/var/lib/kubelet/pods/84745d7f-a7ab-4124-b06c-69b2263b63a6/volumes" Jul 07 15:06:12.915489 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:06:12.915454 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:06:12.919834 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:06:12.919810 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-75b4cf9bdb-65swh" Jul 07 15:06:17.945185 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:06:17.945097 2556 generic.go:356] "Generic (PLEG): container finished" podID="21e316ac-2667-4c4e-b726-88a3c866eb77" containerID="f240fa6c9419318ace4293cf831207ea6413b1d2c5a05a88ec6070ac3ca6de71" exitCode=0 Jul 07 15:06:17.945664 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:06:17.945175 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wc9cn" event={"ID":"21e316ac-2667-4c4e-b726-88a3c866eb77","Type":"ContainerDied","Data":"f240fa6c9419318ace4293cf831207ea6413b1d2c5a05a88ec6070ac3ca6de71"} Jul 07 15:06:17.945664 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:06:17.945582 2556 scope.go:117] "RemoveContainer" containerID="f240fa6c9419318ace4293cf831207ea6413b1d2c5a05a88ec6070ac3ca6de71" Jul 07 15:06:18.950784 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:06:18.950748 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wc9cn" event={"ID":"21e316ac-2667-4c4e-b726-88a3c866eb77","Type":"ContainerStarted","Data":"e938267516e0be45772b1020b35a4b0d0f71c5c025225d3965d71eea2fabb723"} Jul 07 15:06:22.967766 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:06:22.967677 2556 generic.go:356] "Generic (PLEG): container finished" podID="89ce2379-8dcb-4fde-8423-47dc9bf4512f" containerID="d9cf85011ec339d313076a680773c9943e305b38acc7d90cdef96e4603211484" exitCode=0 Jul 07 15:06:22.967766 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:06:22.967740 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" event={"ID":"89ce2379-8dcb-4fde-8423-47dc9bf4512f","Type":"ContainerDied","Data":"d9cf85011ec339d313076a680773c9943e305b38acc7d90cdef96e4603211484"} Jul 07 15:06:22.968270 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:06:22.968088 2556 scope.go:117] "RemoveContainer" containerID="d9cf85011ec339d313076a680773c9943e305b38acc7d90cdef96e4603211484" Jul 07 15:06:23.974058 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:06:23.974021 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-7qqkl" event={"ID":"89ce2379-8dcb-4fde-8423-47dc9bf4512f","Type":"ContainerStarted","Data":"19b84f8a46d0e18d0fa0b56159a3a8ffcd62c70155c99a156580e491d579926b"} Jul 07 15:06:34.767501 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:06:34.767460 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:06:34.783392 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:06:34.783365 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:06:35.033414 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:06:35.033334 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 15:07:06.702375 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:06.702335 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cd7b76fbd-ttm4h"] Jul 07 15:07:31.724414 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:31.724366 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6cd7b76fbd-ttm4h" podUID="f5ba963a-f45b-412e-abbd-ba8c79f6324e" containerName="console" containerID="cri-o://427660e54bf0a97fbc88a089143614f85f03a29662f376829f8710cb4be84875" gracePeriod=15 Jul 07 15:07:31.981089 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:31.981027 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:07:32.164304 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.164264 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-oauth-serving-cert\") pod \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " Jul 07 15:07:32.164481 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.164322 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5ba963a-f45b-412e-abbd-ba8c79f6324e-console-oauth-config\") pod \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " Jul 07 15:07:32.164481 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.164345 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-trusted-ca-bundle\") pod \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " Jul 07 15:07:32.164481 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.164368 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-service-ca\") pod \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " Jul 07 15:07:32.164481 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.164401 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbbrh\" (UniqueName: \"kubernetes.io/projected/f5ba963a-f45b-412e-abbd-ba8c79f6324e-kube-api-access-bbbrh\") pod \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " Jul 07 15:07:32.164481 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.164435 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5ba963a-f45b-412e-abbd-ba8c79f6324e-console-serving-cert\") pod \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " Jul 07 15:07:32.164714 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.164515 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-console-config\") pod \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\" (UID: \"f5ba963a-f45b-412e-abbd-ba8c79f6324e\") " Jul 07 15:07:32.164839 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.164796 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f5ba963a-f45b-412e-abbd-ba8c79f6324e" (UID: "f5ba963a-f45b-412e-abbd-ba8c79f6324e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:07:32.164981 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.164942 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-console-config" (OuterVolumeSpecName: "console-config") pod "f5ba963a-f45b-412e-abbd-ba8c79f6324e" (UID: "f5ba963a-f45b-412e-abbd-ba8c79f6324e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:07:32.165071 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.165044 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f5ba963a-f45b-412e-abbd-ba8c79f6324e" (UID: "f5ba963a-f45b-412e-abbd-ba8c79f6324e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:07:32.165206 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.165189 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-service-ca" (OuterVolumeSpecName: "service-ca") pod "f5ba963a-f45b-412e-abbd-ba8c79f6324e" (UID: "f5ba963a-f45b-412e-abbd-ba8c79f6324e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:07:32.166487 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.166456 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5ba963a-f45b-412e-abbd-ba8c79f6324e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f5ba963a-f45b-412e-abbd-ba8c79f6324e" (UID: "f5ba963a-f45b-412e-abbd-ba8c79f6324e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:07:32.166576 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.166549 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5ba963a-f45b-412e-abbd-ba8c79f6324e-kube-api-access-bbbrh" (OuterVolumeSpecName: "kube-api-access-bbbrh") pod "f5ba963a-f45b-412e-abbd-ba8c79f6324e" (UID: "f5ba963a-f45b-412e-abbd-ba8c79f6324e"). InnerVolumeSpecName "kube-api-access-bbbrh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:07:32.166616 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.166578 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5ba963a-f45b-412e-abbd-ba8c79f6324e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f5ba963a-f45b-412e-abbd-ba8c79f6324e" (UID: "f5ba963a-f45b-412e-abbd-ba8c79f6324e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:07:32.245098 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.245007 2556 generic.go:356] "Generic (PLEG): container finished" podID="f5ba963a-f45b-412e-abbd-ba8c79f6324e" containerID="427660e54bf0a97fbc88a089143614f85f03a29662f376829f8710cb4be84875" exitCode=0 Jul 07 15:07:32.245098 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.245055 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6cd7b76fbd-ttm4h" Jul 07 15:07:32.245098 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.245077 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd7b76fbd-ttm4h" event={"ID":"f5ba963a-f45b-412e-abbd-ba8c79f6324e","Type":"ContainerDied","Data":"427660e54bf0a97fbc88a089143614f85f03a29662f376829f8710cb4be84875"} Jul 07 15:07:32.245303 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.245117 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6cd7b76fbd-ttm4h" event={"ID":"f5ba963a-f45b-412e-abbd-ba8c79f6324e","Type":"ContainerDied","Data":"275db41af8fbb1e23c57b60ab4053b73cca67a8384fc1bfc8f3a0508573cb8e8"} Jul 07 15:07:32.245303 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.245142 2556 scope.go:117] "RemoveContainer" containerID="427660e54bf0a97fbc88a089143614f85f03a29662f376829f8710cb4be84875" Jul 07 15:07:32.254689 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.254671 2556 scope.go:117] "RemoveContainer" containerID="427660e54bf0a97fbc88a089143614f85f03a29662f376829f8710cb4be84875" Jul 07 15:07:32.255002 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:07:32.254983 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"427660e54bf0a97fbc88a089143614f85f03a29662f376829f8710cb4be84875\": container with ID starting with 427660e54bf0a97fbc88a089143614f85f03a29662f376829f8710cb4be84875 not found: ID does not exist" containerID="427660e54bf0a97fbc88a089143614f85f03a29662f376829f8710cb4be84875" Jul 07 15:07:32.255064 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.255011 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"427660e54bf0a97fbc88a089143614f85f03a29662f376829f8710cb4be84875"} err="failed to get container status \"427660e54bf0a97fbc88a089143614f85f03a29662f376829f8710cb4be84875\": rpc error: code = NotFound desc = could not find container \"427660e54bf0a97fbc88a089143614f85f03a29662f376829f8710cb4be84875\": container with ID starting with 427660e54bf0a97fbc88a089143614f85f03a29662f376829f8710cb4be84875 not found: ID does not exist" Jul 07 15:07:32.265749 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.265725 2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f5ba963a-f45b-412e-abbd-ba8c79f6324e-console-serving-cert\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:07:32.265749 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.265747 2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-console-config\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:07:32.265901 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.265756 2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-oauth-serving-cert\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:07:32.265901 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.265764 2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f5ba963a-f45b-412e-abbd-ba8c79f6324e-console-oauth-config\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:07:32.265901 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.265774 2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-trusted-ca-bundle\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:07:32.265901 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.265782 2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f5ba963a-f45b-412e-abbd-ba8c79f6324e-service-ca\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:07:32.265901 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.265789 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbbrh\" (UniqueName: \"kubernetes.io/projected/f5ba963a-f45b-412e-abbd-ba8c79f6324e-kube-api-access-bbbrh\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:07:32.266249 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.266229 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6cd7b76fbd-ttm4h"] Jul 07 15:07:32.270394 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:32.270373 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6cd7b76fbd-ttm4h"] Jul 07 15:07:33.787613 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:07:33.787576 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5ba963a-f45b-412e-abbd-ba8c79f6324e" path="/var/lib/kubelet/pods/f5ba963a-f45b-412e-abbd-ba8c79f6324e/volumes" Jul 07 15:08:54.538223 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.538192 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6"] Jul 07 15:08:54.538731 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.538677 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84745d7f-a7ab-4124-b06c-69b2263b63a6" containerName="console" Jul 07 15:08:54.538731 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.538697 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="84745d7f-a7ab-4124-b06c-69b2263b63a6" containerName="console" Jul 07 15:08:54.538731 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.538715 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5ba963a-f45b-412e-abbd-ba8c79f6324e" containerName="console" Jul 07 15:08:54.538731 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.538724 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5ba963a-f45b-412e-abbd-ba8c79f6324e" containerName="console" Jul 07 15:08:54.538966 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.538822 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="84745d7f-a7ab-4124-b06c-69b2263b63a6" containerName="console" Jul 07 15:08:54.538966 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.538837 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5ba963a-f45b-412e-abbd-ba8c79f6324e" containerName="console" Jul 07 15:08:54.541107 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.541080 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" Jul 07 15:08:54.544256 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.544226 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 15:08:54.544496 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.544475 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 15:08:54.544580 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.544559 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 15:08:54.544745 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.544706 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-f2t5w\"" Jul 07 15:08:54.544889 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.544751 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 15:08:54.557726 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.557693 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6"] Jul 07 15:08:54.677611 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.677575 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/728e97a8-e828-46f0-bb12-6aa37b215e9a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6\" (UID: \"728e97a8-e828-46f0-bb12-6aa37b215e9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" Jul 07 15:08:54.677824 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.677627 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/728e97a8-e828-46f0-bb12-6aa37b215e9a-webhook-cert\") pod \"opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6\" (UID: \"728e97a8-e828-46f0-bb12-6aa37b215e9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" Jul 07 15:08:54.677824 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.677659 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jj6sv\" (UniqueName: \"kubernetes.io/projected/728e97a8-e828-46f0-bb12-6aa37b215e9a-kube-api-access-jj6sv\") pod \"opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6\" (UID: \"728e97a8-e828-46f0-bb12-6aa37b215e9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" Jul 07 15:08:54.677824 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.677757 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/728e97a8-e828-46f0-bb12-6aa37b215e9a-tmp\") pod \"opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6\" (UID: \"728e97a8-e828-46f0-bb12-6aa37b215e9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" Jul 07 15:08:54.677824 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.677810 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/728e97a8-e828-46f0-bb12-6aa37b215e9a-manifests\") pod \"opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6\" (UID: \"728e97a8-e828-46f0-bb12-6aa37b215e9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" Jul 07 15:08:54.778335 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.778301 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/728e97a8-e828-46f0-bb12-6aa37b215e9a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6\" (UID: \"728e97a8-e828-46f0-bb12-6aa37b215e9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" Jul 07 15:08:54.778519 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.778359 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/728e97a8-e828-46f0-bb12-6aa37b215e9a-webhook-cert\") pod \"opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6\" (UID: \"728e97a8-e828-46f0-bb12-6aa37b215e9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" Jul 07 15:08:54.778519 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.778390 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jj6sv\" (UniqueName: \"kubernetes.io/projected/728e97a8-e828-46f0-bb12-6aa37b215e9a-kube-api-access-jj6sv\") pod \"opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6\" (UID: \"728e97a8-e828-46f0-bb12-6aa37b215e9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" Jul 07 15:08:54.778519 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.778439 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/728e97a8-e828-46f0-bb12-6aa37b215e9a-tmp\") pod \"opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6\" (UID: \"728e97a8-e828-46f0-bb12-6aa37b215e9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" Jul 07 15:08:54.778519 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.778486 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/728e97a8-e828-46f0-bb12-6aa37b215e9a-manifests\") pod \"opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6\" (UID: \"728e97a8-e828-46f0-bb12-6aa37b215e9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" Jul 07 15:08:54.778980 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.778956 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/728e97a8-e828-46f0-bb12-6aa37b215e9a-manifests\") pod \"opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6\" (UID: \"728e97a8-e828-46f0-bb12-6aa37b215e9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" Jul 07 15:08:54.780788 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.780763 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/728e97a8-e828-46f0-bb12-6aa37b215e9a-webhook-cert\") pod \"opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6\" (UID: \"728e97a8-e828-46f0-bb12-6aa37b215e9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" Jul 07 15:08:54.781013 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.780988 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/728e97a8-e828-46f0-bb12-6aa37b215e9a-tmp\") pod \"opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6\" (UID: \"728e97a8-e828-46f0-bb12-6aa37b215e9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" Jul 07 15:08:54.781342 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.781322 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/728e97a8-e828-46f0-bb12-6aa37b215e9a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6\" (UID: \"728e97a8-e828-46f0-bb12-6aa37b215e9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" Jul 07 15:08:54.796782 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.796725 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jj6sv\" (UniqueName: \"kubernetes.io/projected/728e97a8-e828-46f0-bb12-6aa37b215e9a-kube-api-access-jj6sv\") pod \"opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6\" (UID: \"728e97a8-e828-46f0-bb12-6aa37b215e9a\") " pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" Jul 07 15:08:54.861578 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.861547 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" Jul 07 15:08:54.999920 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:54.999880 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6"] Jul 07 15:08:55.003784 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:08:55.003755 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod728e97a8_e828_46f0_bb12_6aa37b215e9a.slice/crio-9e14c13e89a8cf8ec976f155cdebeb93b27d7aba2361a212ff8612ade0629a0f WatchSource:0}: Error finding container 9e14c13e89a8cf8ec976f155cdebeb93b27d7aba2361a212ff8612ade0629a0f: Status 404 returned error can't find the container with id 9e14c13e89a8cf8ec976f155cdebeb93b27d7aba2361a212ff8612ade0629a0f Jul 07 15:08:55.005551 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:55.005535 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 15:08:55.581883 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:55.581845 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" event={"ID":"728e97a8-e828-46f0-bb12-6aa37b215e9a","Type":"ContainerStarted","Data":"9e14c13e89a8cf8ec976f155cdebeb93b27d7aba2361a212ff8612ade0629a0f"} Jul 07 15:08:58.599795 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:58.599760 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" event={"ID":"728e97a8-e828-46f0-bb12-6aa37b215e9a","Type":"ContainerStarted","Data":"e2df7ad4a60903ea6e607f8db48bcad2acea967c0f3f96139b4e72a5dfa68806"} Jul 07 15:08:59.606275 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:59.606237 2556 generic.go:356] "Generic (PLEG): container finished" podID="728e97a8-e828-46f0-bb12-6aa37b215e9a" containerID="e2df7ad4a60903ea6e607f8db48bcad2acea967c0f3f96139b4e72a5dfa68806" exitCode=0 Jul 07 15:08:59.606679 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:08:59.606325 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" event={"ID":"728e97a8-e828-46f0-bb12-6aa37b215e9a","Type":"ContainerDied","Data":"e2df7ad4a60903ea6e607f8db48bcad2acea967c0f3f96139b4e72a5dfa68806"} Jul 07 15:09:00.619655 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:00.619598 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" event={"ID":"728e97a8-e828-46f0-bb12-6aa37b215e9a","Type":"ContainerStarted","Data":"9f44cade1492bd36643a4434a3411815ef19a111ff1ab91c2c2c7e0df83b4d2a"} Jul 07 15:09:00.620118 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:00.619722 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" Jul 07 15:09:00.646973 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:00.646918 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" podStartSLOduration=1.920580406 podStartE2EDuration="6.646896001s" totalImagesPullingTime="3.677816533s" totalInitContainerRuntime="1.048499062s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:08:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:09:00.64530637 +0000 UTC m=+489.461572420" watchObservedRunningTime="2026-07-07 15:09:00.646896001 +0000 UTC m=+489.463162040" Jul 07 15:09:11.625984 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:11.625957 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6" Jul 07 15:09:14.568843 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:14.568813 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-55947775b5-n2mhz"] Jul 07 15:09:14.571723 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:14.571699 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-55947775b5-n2mhz" Jul 07 15:09:14.576510 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:14.576478 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-l26bg\"" Jul 07 15:09:14.576510 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:14.576479 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 15:09:14.576677 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:14.576481 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 15:09:14.576677 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:14.576559 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 15:09:14.576677 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:14.576527 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 15:09:14.576677 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:14.576528 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 15:09:14.587601 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:14.587579 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-55947775b5-n2mhz"] Jul 07 15:09:14.651106 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:14.651074 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gcb5\" (UniqueName: \"kubernetes.io/projected/fdc083d3-a349-401f-ab70-6157042df018-kube-api-access-9gcb5\") pod \"lws-controller-manager-55947775b5-n2mhz\" (UID: \"fdc083d3-a349-401f-ab70-6157042df018\") " pod="openshift-lws-operator/lws-controller-manager-55947775b5-n2mhz" Jul 07 15:09:14.651346 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:14.651198 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fdc083d3-a349-401f-ab70-6157042df018-metrics-cert\") pod \"lws-controller-manager-55947775b5-n2mhz\" (UID: \"fdc083d3-a349-401f-ab70-6157042df018\") " pod="openshift-lws-operator/lws-controller-manager-55947775b5-n2mhz" Jul 07 15:09:14.651346 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:14.651285 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fdc083d3-a349-401f-ab70-6157042df018-manager-config\") pod \"lws-controller-manager-55947775b5-n2mhz\" (UID: \"fdc083d3-a349-401f-ab70-6157042df018\") " pod="openshift-lws-operator/lws-controller-manager-55947775b5-n2mhz" Jul 07 15:09:14.651346 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:14.651321 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fdc083d3-a349-401f-ab70-6157042df018-cert\") pod \"lws-controller-manager-55947775b5-n2mhz\" (UID: \"fdc083d3-a349-401f-ab70-6157042df018\") " pod="openshift-lws-operator/lws-controller-manager-55947775b5-n2mhz" Jul 07 15:09:14.752289 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:14.752254 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gcb5\" (UniqueName: \"kubernetes.io/projected/fdc083d3-a349-401f-ab70-6157042df018-kube-api-access-9gcb5\") pod \"lws-controller-manager-55947775b5-n2mhz\" (UID: \"fdc083d3-a349-401f-ab70-6157042df018\") " pod="openshift-lws-operator/lws-controller-manager-55947775b5-n2mhz" Jul 07 15:09:14.752289 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:14.752296 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fdc083d3-a349-401f-ab70-6157042df018-metrics-cert\") pod \"lws-controller-manager-55947775b5-n2mhz\" (UID: \"fdc083d3-a349-401f-ab70-6157042df018\") " pod="openshift-lws-operator/lws-controller-manager-55947775b5-n2mhz" Jul 07 15:09:14.752480 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:14.752333 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fdc083d3-a349-401f-ab70-6157042df018-manager-config\") pod \"lws-controller-manager-55947775b5-n2mhz\" (UID: \"fdc083d3-a349-401f-ab70-6157042df018\") " pod="openshift-lws-operator/lws-controller-manager-55947775b5-n2mhz" Jul 07 15:09:14.752480 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:14.752351 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fdc083d3-a349-401f-ab70-6157042df018-cert\") pod \"lws-controller-manager-55947775b5-n2mhz\" (UID: \"fdc083d3-a349-401f-ab70-6157042df018\") " pod="openshift-lws-operator/lws-controller-manager-55947775b5-n2mhz" Jul 07 15:09:14.753163 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:14.753138 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fdc083d3-a349-401f-ab70-6157042df018-manager-config\") pod \"lws-controller-manager-55947775b5-n2mhz\" (UID: \"fdc083d3-a349-401f-ab70-6157042df018\") " pod="openshift-lws-operator/lws-controller-manager-55947775b5-n2mhz" Jul 07 15:09:14.754809 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:14.754780 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fdc083d3-a349-401f-ab70-6157042df018-cert\") pod \"lws-controller-manager-55947775b5-n2mhz\" (UID: \"fdc083d3-a349-401f-ab70-6157042df018\") " pod="openshift-lws-operator/lws-controller-manager-55947775b5-n2mhz" Jul 07 15:09:14.754939 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:14.754860 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fdc083d3-a349-401f-ab70-6157042df018-metrics-cert\") pod \"lws-controller-manager-55947775b5-n2mhz\" (UID: \"fdc083d3-a349-401f-ab70-6157042df018\") " pod="openshift-lws-operator/lws-controller-manager-55947775b5-n2mhz" Jul 07 15:09:14.762807 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:14.762776 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gcb5\" (UniqueName: \"kubernetes.io/projected/fdc083d3-a349-401f-ab70-6157042df018-kube-api-access-9gcb5\") pod \"lws-controller-manager-55947775b5-n2mhz\" (UID: \"fdc083d3-a349-401f-ab70-6157042df018\") " pod="openshift-lws-operator/lws-controller-manager-55947775b5-n2mhz" Jul 07 15:09:14.881166 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:14.881127 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-55947775b5-n2mhz" Jul 07 15:09:15.015819 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:15.015784 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-55947775b5-n2mhz"] Jul 07 15:09:15.019028 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:09:15.019003 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdc083d3_a349_401f_ab70_6157042df018.slice/crio-8c555ae9fc5e707ab27cf27c45d795a688a841744eb1ec88e0ad9932cfcd623c WatchSource:0}: Error finding container 8c555ae9fc5e707ab27cf27c45d795a688a841744eb1ec88e0ad9932cfcd623c: Status 404 returned error can't find the container with id 8c555ae9fc5e707ab27cf27c45d795a688a841744eb1ec88e0ad9932cfcd623c Jul 07 15:09:15.683230 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:15.683188 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-55947775b5-n2mhz" event={"ID":"fdc083d3-a349-401f-ab70-6157042df018","Type":"ContainerStarted","Data":"8c555ae9fc5e707ab27cf27c45d795a688a841744eb1ec88e0ad9932cfcd623c"} Jul 07 15:09:17.693005 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:17.692968 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-55947775b5-n2mhz" event={"ID":"fdc083d3-a349-401f-ab70-6157042df018","Type":"ContainerStarted","Data":"9db6774fee93fc092a5c35e42d306226e28f5cacbe06dc42897cc82170c2c3af"} Jul 07 15:09:17.693377 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:17.693158 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-55947775b5-n2mhz" Jul 07 15:09:17.710697 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:17.710647 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-55947775b5-n2mhz" podStartSLOduration=1.20828793 podStartE2EDuration="3.710632299s" totalImagesPullingTime="2.502344369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:09:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:09:17.708671427 +0000 UTC m=+506.524937482" watchObservedRunningTime="2026-07-07 15:09:17.710632299 +0000 UTC m=+506.526898337" Jul 07 15:09:23.601363 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.601327 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6"] Jul 07 15:09:23.604789 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.604772 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:23.607911 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.607890 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 15:09:23.608129 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.608112 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 15:09:23.608224 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.608113 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 15:09:23.608224 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.608163 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-phtwb\"" Jul 07 15:09:23.608681 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.608659 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 15:09:23.623685 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.623656 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6"] Jul 07 15:09:23.730936 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.730898 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/98339dff-159d-40b8-b7f1-81492771670d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-482t6\" (UID: \"98339dff-159d-40b8-b7f1-81492771670d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:23.731122 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.730950 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/98339dff-159d-40b8-b7f1-81492771670d-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-482t6\" (UID: \"98339dff-159d-40b8-b7f1-81492771670d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:23.731122 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.731031 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpqvx\" (UniqueName: \"kubernetes.io/projected/98339dff-159d-40b8-b7f1-81492771670d-kube-api-access-fpqvx\") pod \"istiod-openshift-gateway-94bb8fbfd-482t6\" (UID: \"98339dff-159d-40b8-b7f1-81492771670d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:23.731230 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.731119 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/98339dff-159d-40b8-b7f1-81492771670d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-482t6\" (UID: \"98339dff-159d-40b8-b7f1-81492771670d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:23.731230 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.731207 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98339dff-159d-40b8-b7f1-81492771670d-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-482t6\" (UID: \"98339dff-159d-40b8-b7f1-81492771670d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:23.731324 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.731252 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/98339dff-159d-40b8-b7f1-81492771670d-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-482t6\" (UID: \"98339dff-159d-40b8-b7f1-81492771670d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:23.731324 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.731306 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/98339dff-159d-40b8-b7f1-81492771670d-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-482t6\" (UID: \"98339dff-159d-40b8-b7f1-81492771670d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:23.838694 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.838656 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpqvx\" (UniqueName: \"kubernetes.io/projected/98339dff-159d-40b8-b7f1-81492771670d-kube-api-access-fpqvx\") pod \"istiod-openshift-gateway-94bb8fbfd-482t6\" (UID: \"98339dff-159d-40b8-b7f1-81492771670d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:23.838694 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.838697 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/98339dff-159d-40b8-b7f1-81492771670d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-482t6\" (UID: \"98339dff-159d-40b8-b7f1-81492771670d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:23.838952 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.838813 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98339dff-159d-40b8-b7f1-81492771670d-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-482t6\" (UID: \"98339dff-159d-40b8-b7f1-81492771670d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:23.838952 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.838837 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/98339dff-159d-40b8-b7f1-81492771670d-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-482t6\" (UID: \"98339dff-159d-40b8-b7f1-81492771670d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:23.838952 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.838862 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/98339dff-159d-40b8-b7f1-81492771670d-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-482t6\" (UID: \"98339dff-159d-40b8-b7f1-81492771670d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:23.838952 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.838930 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/98339dff-159d-40b8-b7f1-81492771670d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-482t6\" (UID: \"98339dff-159d-40b8-b7f1-81492771670d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:23.839149 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.838968 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/98339dff-159d-40b8-b7f1-81492771670d-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-482t6\" (UID: \"98339dff-159d-40b8-b7f1-81492771670d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:23.839506 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.839477 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/98339dff-159d-40b8-b7f1-81492771670d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-482t6\" (UID: \"98339dff-159d-40b8-b7f1-81492771670d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:23.841598 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.841566 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/98339dff-159d-40b8-b7f1-81492771670d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-482t6\" (UID: \"98339dff-159d-40b8-b7f1-81492771670d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:23.841698 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.841667 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/98339dff-159d-40b8-b7f1-81492771670d-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-482t6\" (UID: \"98339dff-159d-40b8-b7f1-81492771670d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:23.841698 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.841683 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/98339dff-159d-40b8-b7f1-81492771670d-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-482t6\" (UID: \"98339dff-159d-40b8-b7f1-81492771670d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:23.841777 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.841759 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/98339dff-159d-40b8-b7f1-81492771670d-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-482t6\" (UID: \"98339dff-159d-40b8-b7f1-81492771670d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:23.848566 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.848545 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98339dff-159d-40b8-b7f1-81492771670d-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-482t6\" (UID: \"98339dff-159d-40b8-b7f1-81492771670d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:23.848762 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.848745 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpqvx\" (UniqueName: \"kubernetes.io/projected/98339dff-159d-40b8-b7f1-81492771670d-kube-api-access-fpqvx\") pod \"istiod-openshift-gateway-94bb8fbfd-482t6\" (UID: \"98339dff-159d-40b8-b7f1-81492771670d\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:23.914099 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:23.914019 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:24.058684 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:24.058646 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6"] Jul 07 15:09:24.062835 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:09:24.062806 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98339dff_159d_40b8_b7f1_81492771670d.slice/crio-2511a0a41db14d20ae83d29c0caddb91f02d544610aa7e6cecbd90fd6f2ab0a6 WatchSource:0}: Error finding container 2511a0a41db14d20ae83d29c0caddb91f02d544610aa7e6cecbd90fd6f2ab0a6: Status 404 returned error can't find the container with id 2511a0a41db14d20ae83d29c0caddb91f02d544610aa7e6cecbd90fd6f2ab0a6 Jul 07 15:09:24.724606 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:24.724560 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" event={"ID":"98339dff-159d-40b8-b7f1-81492771670d","Type":"ContainerStarted","Data":"2511a0a41db14d20ae83d29c0caddb91f02d544610aa7e6cecbd90fd6f2ab0a6"} Jul 07 15:09:26.679713 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:26.679670 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:09:26.680027 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:26.679739 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:09:27.747293 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:27.747244 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" event={"ID":"98339dff-159d-40b8-b7f1-81492771670d","Type":"ContainerStarted","Data":"e2ba7e9208a6b06172546a8d536a446fba58497b41251c40bbb76b4b1e9f9d5d"} Jul 07 15:09:27.748410 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:27.748359 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:27.750288 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:27.750262 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" Jul 07 15:09:27.904332 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:27.904265 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-482t6" podStartSLOduration=2.28993392 podStartE2EDuration="4.904243756s" totalImagesPullingTime="2.614309836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:09:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:09:27.848399034 +0000 UTC m=+516.664665083" watchObservedRunningTime="2026-07-07 15:09:27.904243756 +0000 UTC m=+516.720509795" Jul 07 15:09:28.700705 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:09:28.700677 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-55947775b5-n2mhz" Jul 07 15:10:51.609039 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:10:51.609008 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/1.log" Jul 07 15:10:51.610775 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:10:51.610751 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/1.log" Jul 07 15:11:22.206813 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:11:22.206780 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zwm9m"] Jul 07 15:11:22.210522 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:11:22.210496 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zwm9m" Jul 07 15:11:22.213274 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:11:22.213252 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 15:11:22.213413 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:11:22.213291 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 15:11:22.214124 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:11:22.214108 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-5zzdl\"" Jul 07 15:11:22.220917 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:11:22.220890 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zwm9m"] Jul 07 15:11:22.314333 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:11:22.314292 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6wvf\" (UniqueName: \"kubernetes.io/projected/2bf1b387-02dd-4ef5-9850-ab3e648c2899-kube-api-access-q6wvf\") pod \"limitador-operator-controller-manager-859d7b78bd-zwm9m\" (UID: \"2bf1b387-02dd-4ef5-9850-ab3e648c2899\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zwm9m" Jul 07 15:11:22.415364 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:11:22.415326 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6wvf\" (UniqueName: \"kubernetes.io/projected/2bf1b387-02dd-4ef5-9850-ab3e648c2899-kube-api-access-q6wvf\") pod \"limitador-operator-controller-manager-859d7b78bd-zwm9m\" (UID: \"2bf1b387-02dd-4ef5-9850-ab3e648c2899\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zwm9m" Jul 07 15:11:22.428720 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:11:22.428695 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6wvf\" (UniqueName: \"kubernetes.io/projected/2bf1b387-02dd-4ef5-9850-ab3e648c2899-kube-api-access-q6wvf\") pod \"limitador-operator-controller-manager-859d7b78bd-zwm9m\" (UID: \"2bf1b387-02dd-4ef5-9850-ab3e648c2899\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zwm9m" Jul 07 15:11:22.522673 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:11:22.522589 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zwm9m" Jul 07 15:11:22.665137 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:11:22.665113 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zwm9m"] Jul 07 15:11:22.667705 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:11:22.667672 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bf1b387_02dd_4ef5_9850_ab3e648c2899.slice/crio-b713ff6cabab66c812abb09e665dd1adf0ed5b7c09fd5ef06f82f4c5395828b9 WatchSource:0}: Error finding container b713ff6cabab66c812abb09e665dd1adf0ed5b7c09fd5ef06f82f4c5395828b9: Status 404 returned error can't find the container with id b713ff6cabab66c812abb09e665dd1adf0ed5b7c09fd5ef06f82f4c5395828b9 Jul 07 15:11:23.252448 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:11:23.252414 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zwm9m" event={"ID":"2bf1b387-02dd-4ef5-9850-ab3e648c2899","Type":"ContainerStarted","Data":"b713ff6cabab66c812abb09e665dd1adf0ed5b7c09fd5ef06f82f4c5395828b9"} Jul 07 15:11:25.264856 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:11:25.264821 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zwm9m" event={"ID":"2bf1b387-02dd-4ef5-9850-ab3e648c2899","Type":"ContainerStarted","Data":"f019f8461d6874c2fb402363c82620ec56c9407d523e37f36a43b671d0e8d1e4"} Jul 07 15:11:25.265259 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:11:25.264955 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zwm9m" Jul 07 15:11:25.285246 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:11:25.285197 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zwm9m" podStartSLOduration=0.869203036 podStartE2EDuration="3.285180706s" totalImagesPullingTime="2.41597767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:11:25.28408491 +0000 UTC m=+634.100350943" watchObservedRunningTime="2026-07-07 15:11:25.285180706 +0000 UTC m=+634.101446744" Jul 07 15:11:36.272289 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:11:36.272258 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-zwm9m" Jul 07 15:13:26.291853 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:13:26.291824 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-qn6gm"] Jul 07 15:13:26.295028 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:13:26.295012 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-qn6gm" Jul 07 15:13:26.297733 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:13:26.297712 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 07 15:13:26.297860 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:13:26.297841 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-ghgh9\"" Jul 07 15:13:26.307133 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:13:26.307111 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-qn6gm"] Jul 07 15:13:26.340634 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:13:26.340606 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klw8x\" (UniqueName: \"kubernetes.io/projected/117c8485-434b-4a08-9a53-4883af58af7a-kube-api-access-klw8x\") pod \"payload-processing-58f86c7798-qn6gm\" (UID: \"117c8485-434b-4a08-9a53-4883af58af7a\") " pod="openshift-ingress/payload-processing-58f86c7798-qn6gm" Jul 07 15:13:26.340787 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:13:26.340646 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/117c8485-434b-4a08-9a53-4883af58af7a-plugins-config-volume\") pod \"payload-processing-58f86c7798-qn6gm\" (UID: \"117c8485-434b-4a08-9a53-4883af58af7a\") " pod="openshift-ingress/payload-processing-58f86c7798-qn6gm" Jul 07 15:13:26.441782 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:13:26.441743 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klw8x\" (UniqueName: \"kubernetes.io/projected/117c8485-434b-4a08-9a53-4883af58af7a-kube-api-access-klw8x\") pod \"payload-processing-58f86c7798-qn6gm\" (UID: \"117c8485-434b-4a08-9a53-4883af58af7a\") " pod="openshift-ingress/payload-processing-58f86c7798-qn6gm" Jul 07 15:13:26.441967 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:13:26.441860 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/117c8485-434b-4a08-9a53-4883af58af7a-plugins-config-volume\") pod \"payload-processing-58f86c7798-qn6gm\" (UID: \"117c8485-434b-4a08-9a53-4883af58af7a\") " pod="openshift-ingress/payload-processing-58f86c7798-qn6gm" Jul 07 15:13:26.442677 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:13:26.442655 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/117c8485-434b-4a08-9a53-4883af58af7a-plugins-config-volume\") pod \"payload-processing-58f86c7798-qn6gm\" (UID: \"117c8485-434b-4a08-9a53-4883af58af7a\") " pod="openshift-ingress/payload-processing-58f86c7798-qn6gm" Jul 07 15:13:26.453089 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:13:26.453067 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klw8x\" (UniqueName: \"kubernetes.io/projected/117c8485-434b-4a08-9a53-4883af58af7a-kube-api-access-klw8x\") pod \"payload-processing-58f86c7798-qn6gm\" (UID: \"117c8485-434b-4a08-9a53-4883af58af7a\") " pod="openshift-ingress/payload-processing-58f86c7798-qn6gm" Jul 07 15:13:26.606510 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:13:26.606475 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-qn6gm" Jul 07 15:13:26.738236 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:13:26.738213 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-qn6gm"] Jul 07 15:13:26.740733 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:13:26.740702 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod117c8485_434b_4a08_9a53_4883af58af7a.slice/crio-8f1875c040f96fce99ac3602149257ed9872dbb1212709f060ae7b782951ee44 WatchSource:0}: Error finding container 8f1875c040f96fce99ac3602149257ed9872dbb1212709f060ae7b782951ee44: Status 404 returned error can't find the container with id 8f1875c040f96fce99ac3602149257ed9872dbb1212709f060ae7b782951ee44 Jul 07 15:13:26.822992 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:13:26.822950 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-qn6gm" event={"ID":"117c8485-434b-4a08-9a53-4883af58af7a","Type":"ContainerStarted","Data":"8f1875c040f96fce99ac3602149257ed9872dbb1212709f060ae7b782951ee44"} Jul 07 15:13:28.832673 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:13:28.832635 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-qn6gm" event={"ID":"117c8485-434b-4a08-9a53-4883af58af7a","Type":"ContainerStarted","Data":"090186a44c35a4f168fa3801e223b66f3f5b3aeeb0115c8ca3324f2a89526c83"} Jul 07 15:13:28.833074 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:13:28.832723 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-qn6gm" Jul 07 15:13:28.849699 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:13:28.849650 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-qn6gm" podStartSLOduration=1.218664486 podStartE2EDuration="2.84963479s" totalImagesPullingTime="1.630970304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:13:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:13:28.847717216 +0000 UTC m=+757.663983255" watchObservedRunningTime="2026-07-07 15:13:28.84963479 +0000 UTC m=+757.665900828" Jul 07 15:13:39.840444 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:13:39.840406 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-qn6gm" Jul 07 15:14:27.003553 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.003521 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx"] Jul 07 15:14:27.006670 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.006650 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:27.008967 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.008949 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 15:14:27.009770 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.009740 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 07 15:14:27.009890 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.009774 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 15:14:27.009890 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.009786 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-shgqg\"" Jul 07 15:14:27.016734 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.016714 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx"] Jul 07 15:14:27.104105 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.104074 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/765b5121-c00d-454e-b5c7-e572ec300fe0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5gjlx\" (UID: \"765b5121-c00d-454e-b5c7-e572ec300fe0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:27.104283 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.104113 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tzxq\" (UniqueName: \"kubernetes.io/projected/765b5121-c00d-454e-b5c7-e572ec300fe0-kube-api-access-6tzxq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5gjlx\" (UID: \"765b5121-c00d-454e-b5c7-e572ec300fe0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:27.104283 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.104151 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/765b5121-c00d-454e-b5c7-e572ec300fe0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5gjlx\" (UID: \"765b5121-c00d-454e-b5c7-e572ec300fe0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:27.104283 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.104178 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/765b5121-c00d-454e-b5c7-e572ec300fe0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5gjlx\" (UID: \"765b5121-c00d-454e-b5c7-e572ec300fe0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:27.104283 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.104245 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/765b5121-c00d-454e-b5c7-e572ec300fe0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5gjlx\" (UID: \"765b5121-c00d-454e-b5c7-e572ec300fe0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:27.104423 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.104283 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/765b5121-c00d-454e-b5c7-e572ec300fe0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5gjlx\" (UID: \"765b5121-c00d-454e-b5c7-e572ec300fe0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:27.204651 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.204614 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/765b5121-c00d-454e-b5c7-e572ec300fe0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5gjlx\" (UID: \"765b5121-c00d-454e-b5c7-e572ec300fe0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:27.204840 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.204659 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/765b5121-c00d-454e-b5c7-e572ec300fe0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5gjlx\" (UID: \"765b5121-c00d-454e-b5c7-e572ec300fe0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:27.204840 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.204697 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/765b5121-c00d-454e-b5c7-e572ec300fe0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5gjlx\" (UID: \"765b5121-c00d-454e-b5c7-e572ec300fe0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:27.204840 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.204713 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/765b5121-c00d-454e-b5c7-e572ec300fe0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5gjlx\" (UID: \"765b5121-c00d-454e-b5c7-e572ec300fe0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:27.204840 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.204735 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/765b5121-c00d-454e-b5c7-e572ec300fe0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5gjlx\" (UID: \"765b5121-c00d-454e-b5c7-e572ec300fe0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:27.204840 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.204760 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tzxq\" (UniqueName: \"kubernetes.io/projected/765b5121-c00d-454e-b5c7-e572ec300fe0-kube-api-access-6tzxq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5gjlx\" (UID: \"765b5121-c00d-454e-b5c7-e572ec300fe0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:27.205192 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.205089 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/765b5121-c00d-454e-b5c7-e572ec300fe0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5gjlx\" (UID: \"765b5121-c00d-454e-b5c7-e572ec300fe0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:27.205192 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.205153 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/765b5121-c00d-454e-b5c7-e572ec300fe0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5gjlx\" (UID: \"765b5121-c00d-454e-b5c7-e572ec300fe0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:27.205192 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.205176 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/765b5121-c00d-454e-b5c7-e572ec300fe0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5gjlx\" (UID: \"765b5121-c00d-454e-b5c7-e572ec300fe0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:27.207119 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.207098 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/765b5121-c00d-454e-b5c7-e572ec300fe0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5gjlx\" (UID: \"765b5121-c00d-454e-b5c7-e572ec300fe0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:27.207338 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.207321 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/765b5121-c00d-454e-b5c7-e572ec300fe0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5gjlx\" (UID: \"765b5121-c00d-454e-b5c7-e572ec300fe0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:27.213774 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.213748 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tzxq\" (UniqueName: \"kubernetes.io/projected/765b5121-c00d-454e-b5c7-e572ec300fe0-kube-api-access-6tzxq\") pod \"e2e-trlp-test-simulated-kserve-598466548d-5gjlx\" (UID: \"765b5121-c00d-454e-b5c7-e572ec300fe0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:27.317972 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.317862 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:27.452690 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.452665 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx"] Jul 07 15:14:27.454809 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:14:27.454781 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod765b5121_c00d_454e_b5c7_e572ec300fe0.slice/crio-7d1116b634d285b3e5e02a99caf8e6d77ba8ebebb59d3e2e83f75dbe68f6f4dc WatchSource:0}: Error finding container 7d1116b634d285b3e5e02a99caf8e6d77ba8ebebb59d3e2e83f75dbe68f6f4dc: Status 404 returned error can't find the container with id 7d1116b634d285b3e5e02a99caf8e6d77ba8ebebb59d3e2e83f75dbe68f6f4dc Jul 07 15:14:27.456172 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:27.456154 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 15:14:28.094515 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:28.094479 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" event={"ID":"765b5121-c00d-454e-b5c7-e572ec300fe0","Type":"ContainerStarted","Data":"7d1116b634d285b3e5e02a99caf8e6d77ba8ebebb59d3e2e83f75dbe68f6f4dc"} Jul 07 15:14:29.102799 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:29.102766 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" event={"ID":"765b5121-c00d-454e-b5c7-e572ec300fe0","Type":"ContainerStarted","Data":"005652b09ff70f5937bab32f8bd853919f99abbc440b847c968e1f08d80c3176"} Jul 07 15:14:29.120703 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:29.120649 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" podStartSLOduration=2.00742816 podStartE2EDuration="3.120633207s" totalImagesPullingTime="1.113205047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:14:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:14:29.119196983 +0000 UTC m=+817.935463024" watchObservedRunningTime="2026-07-07 15:14:29.120633207 +0000 UTC m=+817.936899254" Jul 07 15:14:37.318604 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:37.318563 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:37.319168 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:37.318638 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:37.331366 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:37.331343 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:14:38.156532 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:14:38.156503 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-5gjlx" Jul 07 15:15:00.135278 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:00.135242 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723955-8wgxz"] Jul 07 15:15:00.137907 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:00.137885 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723955-8wgxz" Jul 07 15:15:00.140040 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:00.140018 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-td6sd\"" Jul 07 15:15:00.145728 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:00.145705 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723955-8wgxz"] Jul 07 15:15:00.309522 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:00.309484 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g65pn\" (UniqueName: \"kubernetes.io/projected/3bca2b6e-9022-4c85-a582-e89a7d8f55fc-kube-api-access-g65pn\") pod \"maas-api-key-cleanup-29723955-8wgxz\" (UID: \"3bca2b6e-9022-4c85-a582-e89a7d8f55fc\") " pod="opendatahub/maas-api-key-cleanup-29723955-8wgxz" Jul 07 15:15:00.410910 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:00.410804 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g65pn\" (UniqueName: \"kubernetes.io/projected/3bca2b6e-9022-4c85-a582-e89a7d8f55fc-kube-api-access-g65pn\") pod \"maas-api-key-cleanup-29723955-8wgxz\" (UID: \"3bca2b6e-9022-4c85-a582-e89a7d8f55fc\") " pod="opendatahub/maas-api-key-cleanup-29723955-8wgxz" Jul 07 15:15:00.419741 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:00.419707 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g65pn\" (UniqueName: \"kubernetes.io/projected/3bca2b6e-9022-4c85-a582-e89a7d8f55fc-kube-api-access-g65pn\") pod \"maas-api-key-cleanup-29723955-8wgxz\" (UID: \"3bca2b6e-9022-4c85-a582-e89a7d8f55fc\") " pod="opendatahub/maas-api-key-cleanup-29723955-8wgxz" Jul 07 15:15:00.449608 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:00.449583 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723955-8wgxz" Jul 07 15:15:00.581079 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:00.581046 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723955-8wgxz"] Jul 07 15:15:00.583857 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:15:00.583833 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bca2b6e_9022_4c85_a582_e89a7d8f55fc.slice/crio-8b5c6a91c98df07c6d18663e11f71e09c1d5c9c3ce17e1fa4e897ac1b2ac2ab9 WatchSource:0}: Error finding container 8b5c6a91c98df07c6d18663e11f71e09c1d5c9c3ce17e1fa4e897ac1b2ac2ab9: Status 404 returned error can't find the container with id 8b5c6a91c98df07c6d18663e11f71e09c1d5c9c3ce17e1fa4e897ac1b2ac2ab9 Jul 07 15:15:01.245265 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:01.245228 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723955-8wgxz" event={"ID":"3bca2b6e-9022-4c85-a582-e89a7d8f55fc","Type":"ContainerStarted","Data":"8b5c6a91c98df07c6d18663e11f71e09c1d5c9c3ce17e1fa4e897ac1b2ac2ab9"} Jul 07 15:15:04.262401 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:04.262366 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723955-8wgxz" event={"ID":"3bca2b6e-9022-4c85-a582-e89a7d8f55fc","Type":"ContainerStarted","Data":"f67e1f9e96b4edcfb5a4168b207eac171d93dcf4f6fca79ed840f8a0c27f2ba8"} Jul 07 15:15:04.276989 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:04.276828 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29723955-8wgxz" podStartSLOduration=1.017456867 podStartE2EDuration="4.276807953s" totalImagesPullingTime="3.259351086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:15:04.276228651 +0000 UTC m=+853.092494691" watchObservedRunningTime="2026-07-07 15:15:04.276807953 +0000 UTC m=+853.093073996" Jul 07 15:15:05.268300 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:05.268269 2556 generic.go:356] "Generic (PLEG): container finished" podID="3bca2b6e-9022-4c85-a582-e89a7d8f55fc" containerID="f67e1f9e96b4edcfb5a4168b207eac171d93dcf4f6fca79ed840f8a0c27f2ba8" exitCode=0 Jul 07 15:15:05.268687 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:05.268339 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723955-8wgxz" event={"ID":"3bca2b6e-9022-4c85-a582-e89a7d8f55fc","Type":"ContainerDied","Data":"f67e1f9e96b4edcfb5a4168b207eac171d93dcf4f6fca79ed840f8a0c27f2ba8"} Jul 07 15:15:06.392963 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:06.392942 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723955-8wgxz" Jul 07 15:15:06.566955 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:06.566877 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g65pn\" (UniqueName: \"kubernetes.io/projected/3bca2b6e-9022-4c85-a582-e89a7d8f55fc-kube-api-access-g65pn\") pod \"3bca2b6e-9022-4c85-a582-e89a7d8f55fc\" (UID: \"3bca2b6e-9022-4c85-a582-e89a7d8f55fc\") " Jul 07 15:15:06.569096 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:06.569072 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bca2b6e-9022-4c85-a582-e89a7d8f55fc-kube-api-access-g65pn" (OuterVolumeSpecName: "kube-api-access-g65pn") pod "3bca2b6e-9022-4c85-a582-e89a7d8f55fc" (UID: "3bca2b6e-9022-4c85-a582-e89a7d8f55fc"). InnerVolumeSpecName "kube-api-access-g65pn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:15:06.668199 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:06.668164 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g65pn\" (UniqueName: \"kubernetes.io/projected/3bca2b6e-9022-4c85-a582-e89a7d8f55fc-kube-api-access-g65pn\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:15:07.284555 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:07.284519 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723955-8wgxz" event={"ID":"3bca2b6e-9022-4c85-a582-e89a7d8f55fc","Type":"ContainerDied","Data":"8b5c6a91c98df07c6d18663e11f71e09c1d5c9c3ce17e1fa4e897ac1b2ac2ab9"} Jul 07 15:15:07.284555 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:07.284547 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723955-8wgxz" Jul 07 15:15:07.284760 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:07.284592 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8b5c6a91c98df07c6d18663e11f71e09c1d5c9c3ce17e1fa4e897ac1b2ac2ab9" Jul 07 15:15:51.638892 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:51.638848 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/1.log" Jul 07 15:15:51.640917 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:15:51.640896 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/1.log" Jul 07 15:20:51.667231 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:20:51.667204 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/1.log" Jul 07 15:20:51.669696 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:20:51.669673 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/1.log" Jul 07 15:25:51.695275 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:25:51.695226 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/1.log" Jul 07 15:25:51.699176 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:25:51.699155 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/1.log" Jul 07 15:29:53.334820 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.334787 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv"] Jul 07 15:29:53.337248 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.335162 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bca2b6e-9022-4c85-a582-e89a7d8f55fc" containerName="cleanup" Jul 07 15:29:53.337248 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.335174 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bca2b6e-9022-4c85-a582-e89a7d8f55fc" containerName="cleanup" Jul 07 15:29:53.337248 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.335242 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bca2b6e-9022-4c85-a582-e89a7d8f55fc" containerName="cleanup" Jul 07 15:29:53.338222 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.338201 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.342772 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.342748 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-03188eb0-openshift-default-dockercfg-2mswb\"" Jul 07 15:29:53.376770 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.376734 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv"] Jul 07 15:29:53.412241 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.412207 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-data\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.412241 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.412239 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3db6320b-3da5-4cbc-9882-26580e2806d1-istiod-ca-cert\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.412445 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.412261 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-envoy\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.412445 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.412286 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-credential-socket\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.412445 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.412368 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-workload-socket\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.412543 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.412458 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-workload-certs\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.412543 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.412478 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-token\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.412543 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.412518 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-podinfo\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.412543 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.412533 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvc4w\" (UniqueName: \"kubernetes.io/projected/3db6320b-3da5-4cbc-9882-26580e2806d1-kube-api-access-mvc4w\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.513973 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.513930 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-workload-socket\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.514156 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.514012 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-workload-certs\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.514156 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.514036 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-token\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.514156 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.514088 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-podinfo\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.514156 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.514114 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvc4w\" (UniqueName: \"kubernetes.io/projected/3db6320b-3da5-4cbc-9882-26580e2806d1-kube-api-access-mvc4w\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.514156 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.514149 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-data\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.514401 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.514177 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3db6320b-3da5-4cbc-9882-26580e2806d1-istiod-ca-cert\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.514401 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.514257 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-envoy\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.514401 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.514319 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-credential-socket\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.514549 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.514398 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-workload-socket\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.514549 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.514477 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-data\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.514676 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.514655 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-credential-socket\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.514747 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.514703 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-workload-certs\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.514860 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.514843 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3db6320b-3da5-4cbc-9882-26580e2806d1-istiod-ca-cert\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.516558 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.516539 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-envoy\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.516692 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.516675 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-podinfo\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.522576 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.522550 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvc4w\" (UniqueName: \"kubernetes.io/projected/3db6320b-3da5-4cbc-9882-26580e2806d1-kube-api-access-mvc4w\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.522679 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.522578 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-token\") pod \"e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.649322 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.649232 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:53.790052 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.790021 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv"] Jul 07 15:29:53.791210 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:29:53.791183 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3db6320b_3da5_4cbc_9882_26580e2806d1.slice/crio-5fe370a845c585a573868d03ff4f0b15e762d5844ea0dcccebb4a2a59c2718f9 WatchSource:0}: Error finding container 5fe370a845c585a573868d03ff4f0b15e762d5844ea0dcccebb4a2a59c2718f9: Status 404 returned error can't find the container with id 5fe370a845c585a573868d03ff4f0b15e762d5844ea0dcccebb4a2a59c2718f9 Jul 07 15:29:53.793248 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:53.793228 2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 15:29:54.361209 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:54.361170 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" event={"ID":"3db6320b-3da5-4cbc-9882-26580e2806d1","Type":"ContainerStarted","Data":"5fe370a845c585a573868d03ff4f0b15e762d5844ea0dcccebb4a2a59c2718f9"} Jul 07 15:29:56.965177 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:56.965139 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:29:56.965420 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:56.965213 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:29:56.965420 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:56.965242 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:29:57.377732 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:57.377689 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" event={"ID":"3db6320b-3da5-4cbc-9882-26580e2806d1","Type":"ContainerStarted","Data":"7a50da328239df1009017bb9e24e330d70ee102868139abec287356201da2a65"} Jul 07 15:29:57.416216 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:57.416156 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" podStartSLOduration=1.244674745 podStartE2EDuration="4.416135065s" totalImagesPullingTime="3.17146032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:29:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:29:57.411726704 +0000 UTC m=+1746.227992766" watchObservedRunningTime="2026-07-07 15:29:57.416135065 +0000 UTC m=+1746.232401103" Jul 07 15:29:57.650038 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:57.649998 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:58.655669 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:58.655641 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:59.387636 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:59.387609 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:59.388744 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:59.388726 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:29:59.899933 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:29:59.899896 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv"] Jul 07 15:30:00.131471 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:00.131439 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723970-lqpkr"] Jul 07 15:30:00.135077 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:00.135054 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723970-lqpkr" Jul 07 15:30:00.137887 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:00.137848 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-td6sd\"" Jul 07 15:30:00.144112 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:00.144084 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723970-lqpkr"] Jul 07 15:30:00.181528 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:00.181446 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mkp4\" (UniqueName: \"kubernetes.io/projected/2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49-kube-api-access-5mkp4\") pod \"maas-api-key-cleanup-29723970-lqpkr\" (UID: \"2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49\") " pod="opendatahub/maas-api-key-cleanup-29723970-lqpkr" Jul 07 15:30:00.282698 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:00.282657 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mkp4\" (UniqueName: \"kubernetes.io/projected/2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49-kube-api-access-5mkp4\") pod \"maas-api-key-cleanup-29723970-lqpkr\" (UID: \"2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49\") " pod="opendatahub/maas-api-key-cleanup-29723970-lqpkr" Jul 07 15:30:00.292286 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:00.292250 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mkp4\" (UniqueName: \"kubernetes.io/projected/2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49-kube-api-access-5mkp4\") pod \"maas-api-key-cleanup-29723970-lqpkr\" (UID: \"2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49\") " pod="opendatahub/maas-api-key-cleanup-29723970-lqpkr" Jul 07 15:30:00.391997 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:00.391967 2556 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" secret="" err="secret \"e2e-ait-03188eb0-openshift-default-dockercfg-2mswb\" not found" Jul 07 15:30:00.446892 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:00.446776 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723970-lqpkr" Jul 07 15:30:00.588416 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:00.588392 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723970-lqpkr"] Jul 07 15:30:00.591054 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:30:00.591022 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ae2ae33_f844_4d0e_84aa_8ded7d7b8c49.slice/crio-bb8baf5a918e41b0bce8a6ae83e5750457e41a2c5d4bd850cbd7143d7dd9c0bd WatchSource:0}: Error finding container bb8baf5a918e41b0bce8a6ae83e5750457e41a2c5d4bd850cbd7143d7dd9c0bd: Status 404 returned error can't find the container with id bb8baf5a918e41b0bce8a6ae83e5750457e41a2c5d4bd850cbd7143d7dd9c0bd Jul 07 15:30:01.398180 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:01.398144 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723970-lqpkr" event={"ID":"2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49","Type":"ContainerStarted","Data":"eb443b646d02a806af79dee6c5b6e4c38cdfc550a3a8bf0487642bb85db466ca"} Jul 07 15:30:01.398677 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:01.398199 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723970-lqpkr" event={"ID":"2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49","Type":"ContainerStarted","Data":"bb8baf5a918e41b0bce8a6ae83e5750457e41a2c5d4bd850cbd7143d7dd9c0bd"} Jul 07 15:30:01.398677 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:01.398278 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" podUID="3db6320b-3da5-4cbc-9882-26580e2806d1" containerName="istio-proxy" containerID="cri-o://7a50da328239df1009017bb9e24e330d70ee102868139abec287356201da2a65" gracePeriod=30 Jul 07 15:30:01.447819 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:01.447759 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29723970-lqpkr" podStartSLOduration=1.447735991 podStartE2EDuration="1.447735991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:30:01.4191224 +0000 UTC m=+1750.235388439" watchObservedRunningTime="2026-07-07 15:30:01.447735991 +0000 UTC m=+1750.264002027" Jul 07 15:30:02.404471 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:02.404438 2556 generic.go:356] "Generic (PLEG): container finished" podID="2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49" containerID="eb443b646d02a806af79dee6c5b6e4c38cdfc550a3a8bf0487642bb85db466ca" exitCode=0 Jul 07 15:30:02.404826 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:02.404522 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723970-lqpkr" event={"ID":"2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49","Type":"ContainerDied","Data":"eb443b646d02a806af79dee6c5b6e4c38cdfc550a3a8bf0487642bb85db466ca"} Jul 07 15:30:03.540344 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:03.540318 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723970-lqpkr" Jul 07 15:30:03.615555 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:03.615516 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mkp4\" (UniqueName: \"kubernetes.io/projected/2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49-kube-api-access-5mkp4\") pod \"2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49\" (UID: \"2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49\") " Jul 07 15:30:03.617623 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:03.617599 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49-kube-api-access-5mkp4" (OuterVolumeSpecName: "kube-api-access-5mkp4") pod "2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49" (UID: "2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49"). InnerVolumeSpecName "kube-api-access-5mkp4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:30:03.716724 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:03.716690 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mkp4\" (UniqueName: \"kubernetes.io/projected/2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49-kube-api-access-5mkp4\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:30:04.415063 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:04.415028 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723970-lqpkr" Jul 07 15:30:04.415063 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:04.415046 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723970-lqpkr" event={"ID":"2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49","Type":"ContainerDied","Data":"bb8baf5a918e41b0bce8a6ae83e5750457e41a2c5d4bd850cbd7143d7dd9c0bd"} Jul 07 15:30:04.415291 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:04.415099 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb8baf5a918e41b0bce8a6ae83e5750457e41a2c5d4bd850cbd7143d7dd9c0bd" Jul 07 15:30:06.654354 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.654330 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:30:06.745423 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.745338 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-token\") pod \"3db6320b-3da5-4cbc-9882-26580e2806d1\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " Jul 07 15:30:06.745423 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.745388 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-workload-socket\") pod \"3db6320b-3da5-4cbc-9882-26580e2806d1\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " Jul 07 15:30:06.745659 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.745430 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3db6320b-3da5-4cbc-9882-26580e2806d1-istiod-ca-cert\") pod \"3db6320b-3da5-4cbc-9882-26580e2806d1\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " Jul 07 15:30:06.745659 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.745459 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvc4w\" (UniqueName: \"kubernetes.io/projected/3db6320b-3da5-4cbc-9882-26580e2806d1-kube-api-access-mvc4w\") pod \"3db6320b-3da5-4cbc-9882-26580e2806d1\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " Jul 07 15:30:06.745659 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.745492 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-data\") pod \"3db6320b-3da5-4cbc-9882-26580e2806d1\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " Jul 07 15:30:06.745659 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.745548 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-workload-certs\") pod \"3db6320b-3da5-4cbc-9882-26580e2806d1\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " Jul 07 15:30:06.745659 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.745608 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-envoy\") pod \"3db6320b-3da5-4cbc-9882-26580e2806d1\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " Jul 07 15:30:06.745659 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.745651 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-credential-socket\") pod \"3db6320b-3da5-4cbc-9882-26580e2806d1\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " Jul 07 15:30:06.745997 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.745675 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-podinfo\") pod \"3db6320b-3da5-4cbc-9882-26580e2806d1\" (UID: \"3db6320b-3da5-4cbc-9882-26580e2806d1\") " Jul 07 15:30:06.745997 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.745763 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3db6320b-3da5-4cbc-9882-26580e2806d1" (UID: "3db6320b-3da5-4cbc-9882-26580e2806d1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:30:06.745997 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.745839 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-data" (OuterVolumeSpecName: "istio-data") pod "3db6320b-3da5-4cbc-9882-26580e2806d1" (UID: "3db6320b-3da5-4cbc-9882-26580e2806d1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:30:06.745997 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.745934 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3db6320b-3da5-4cbc-9882-26580e2806d1" (UID: "3db6320b-3da5-4cbc-9882-26580e2806d1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:30:06.745997 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.745967 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3db6320b-3da5-4cbc-9882-26580e2806d1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3db6320b-3da5-4cbc-9882-26580e2806d1" (UID: "3db6320b-3da5-4cbc-9882-26580e2806d1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:30:06.746236 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.746078 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3db6320b-3da5-4cbc-9882-26580e2806d1" (UID: "3db6320b-3da5-4cbc-9882-26580e2806d1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:30:06.746363 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.746344 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-workload-certs\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:30:06.746448 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.746392 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-credential-socket\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:30:06.746448 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.746411 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-workload-socket\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:30:06.746448 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.746426 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3db6320b-3da5-4cbc-9882-26580e2806d1-istiod-ca-cert\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:30:06.746448 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.746440 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-data\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:30:06.747892 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.747821 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3db6320b-3da5-4cbc-9882-26580e2806d1-kube-api-access-mvc4w" (OuterVolumeSpecName: "kube-api-access-mvc4w") pod "3db6320b-3da5-4cbc-9882-26580e2806d1" (UID: "3db6320b-3da5-4cbc-9882-26580e2806d1"). InnerVolumeSpecName "kube-api-access-mvc4w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:30:06.748004 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.747964 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3db6320b-3da5-4cbc-9882-26580e2806d1" (UID: "3db6320b-3da5-4cbc-9882-26580e2806d1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:30:06.748049 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.747998 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-token" (OuterVolumeSpecName: "istio-token") pod "3db6320b-3da5-4cbc-9882-26580e2806d1" (UID: "3db6320b-3da5-4cbc-9882-26580e2806d1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:30:06.748049 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.748025 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3db6320b-3da5-4cbc-9882-26580e2806d1" (UID: "3db6320b-3da5-4cbc-9882-26580e2806d1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 15:30:06.847128 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.847078 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-token\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:30:06.847128 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.847123 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvc4w\" (UniqueName: \"kubernetes.io/projected/3db6320b-3da5-4cbc-9882-26580e2806d1-kube-api-access-mvc4w\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:30:06.847128 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.847133 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-envoy\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:30:06.847128 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:06.847142 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3db6320b-3da5-4cbc-9882-26580e2806d1-istio-podinfo\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:30:07.429851 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:07.429818 2556 generic.go:356] "Generic (PLEG): container finished" podID="3db6320b-3da5-4cbc-9882-26580e2806d1" containerID="7a50da328239df1009017bb9e24e330d70ee102868139abec287356201da2a65" exitCode=0 Jul 07 15:30:07.430045 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:07.429893 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" event={"ID":"3db6320b-3da5-4cbc-9882-26580e2806d1","Type":"ContainerDied","Data":"7a50da328239df1009017bb9e24e330d70ee102868139abec287356201da2a65"} Jul 07 15:30:07.430045 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:07.429898 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" Jul 07 15:30:07.430045 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:07.429938 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv" event={"ID":"3db6320b-3da5-4cbc-9882-26580e2806d1","Type":"ContainerDied","Data":"5fe370a845c585a573868d03ff4f0b15e762d5844ea0dcccebb4a2a59c2718f9"} Jul 07 15:30:07.430045 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:07.429962 2556 scope.go:117] "RemoveContainer" containerID="7a50da328239df1009017bb9e24e330d70ee102868139abec287356201da2a65" Jul 07 15:30:07.444926 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:07.444892 2556 scope.go:117] "RemoveContainer" containerID="7a50da328239df1009017bb9e24e330d70ee102868139abec287356201da2a65" Jul 07 15:30:07.445296 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:30:07.445264 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a50da328239df1009017bb9e24e330d70ee102868139abec287356201da2a65\": container with ID starting with 7a50da328239df1009017bb9e24e330d70ee102868139abec287356201da2a65 not found: ID does not exist" containerID="7a50da328239df1009017bb9e24e330d70ee102868139abec287356201da2a65" Jul 07 15:30:07.445380 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:07.445309 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a50da328239df1009017bb9e24e330d70ee102868139abec287356201da2a65"} err="failed to get container status \"7a50da328239df1009017bb9e24e330d70ee102868139abec287356201da2a65\": rpc error: code = NotFound desc = could not find container \"7a50da328239df1009017bb9e24e330d70ee102868139abec287356201da2a65\": container with ID starting with 7a50da328239df1009017bb9e24e330d70ee102868139abec287356201da2a65 not found: ID does not exist" Jul 07 15:30:07.456413 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:07.456383 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv"] Jul 07 15:30:07.463194 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:07.463172 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-03188eb0-openshift-default-6d6f8b6df7-hnscv"] Jul 07 15:30:07.788000 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:07.787910 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3db6320b-3da5-4cbc-9882-26580e2806d1" path="/var/lib/kubelet/pods/3db6320b-3da5-4cbc-9882-26580e2806d1/volumes" Jul 07 15:30:37.694049 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.694018 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb"] Jul 07 15:30:37.694404 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.694353 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49" containerName="cleanup" Jul 07 15:30:37.694404 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.694365 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49" containerName="cleanup" Jul 07 15:30:37.694404 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.694381 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3db6320b-3da5-4cbc-9882-26580e2806d1" containerName="istio-proxy" Jul 07 15:30:37.694404 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.694386 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="3db6320b-3da5-4cbc-9882-26580e2806d1" containerName="istio-proxy" Jul 07 15:30:37.694541 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.694447 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49" containerName="cleanup" Jul 07 15:30:37.694541 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.694458 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="3db6320b-3da5-4cbc-9882-26580e2806d1" containerName="istio-proxy" Jul 07 15:30:37.699458 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.699440 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.702354 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.702327 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-caa09bda-openshift-default-dockercfg-cjkmh\"" Jul 07 15:30:37.710561 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.710533 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb"] Jul 07 15:30:37.730046 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.730022 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-envoy\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.730171 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.730067 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-data\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.730171 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.730088 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-podinfo\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.730171 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.730127 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-workload-socket\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.730171 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.730143 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-credential-socket\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.730303 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.730174 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-workload-certs\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.730303 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.730189 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istiod-ca-cert\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.730303 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.730222 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-token\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.730303 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.730273 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7b5l\" (UniqueName: \"kubernetes.io/projected/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-kube-api-access-d7b5l\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.830983 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.830937 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-token\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.830983 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.830984 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7b5l\" (UniqueName: \"kubernetes.io/projected/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-kube-api-access-d7b5l\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.831246 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.831036 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-envoy\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.831246 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.831119 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-data\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.831246 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.831146 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-podinfo\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.831406 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.831359 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-workload-socket\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.831406 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.831393 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-credential-socket\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.831529 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.831500 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-workload-certs\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.831585 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.831544 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istiod-ca-cert\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.831678 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.831648 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-data\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.831850 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.831726 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-workload-socket\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.831959 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.831778 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-credential-socket\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.832047 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.832026 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-workload-certs\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.832228 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.832211 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istiod-ca-cert\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.834088 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.834057 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-envoy\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.834166 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.834156 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-podinfo\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.839983 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.839954 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-token\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:37.840095 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:37.840080 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7b5l\" (UniqueName: \"kubernetes.io/projected/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-kube-api-access-d7b5l\") pod \"e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:38.011996 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:38.011907 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:38.148393 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:38.148360 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb"] Jul 07 15:30:38.152937 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:30:38.152907 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd39e44e1_4755_4b3a_b3e6_79c4a58ef5ca.slice/crio-a0ceeaf3714ec8282d9900858a084918bc8cb333cd4fbb393449d4a40602aa29 WatchSource:0}: Error finding container a0ceeaf3714ec8282d9900858a084918bc8cb333cd4fbb393449d4a40602aa29: Status 404 returned error can't find the container with id a0ceeaf3714ec8282d9900858a084918bc8cb333cd4fbb393449d4a40602aa29 Jul 07 15:30:38.155040 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:38.155001 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:30:38.155132 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:38.155076 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:30:38.155132 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:38.155107 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:30:38.588816 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:38.588776 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" event={"ID":"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca","Type":"ContainerStarted","Data":"46e1df03aa8a70756ad8b500bc3fa3da4d8d3d8e276ba50b1a9c27246f8902f5"} Jul 07 15:30:38.589017 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:38.588842 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" event={"ID":"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca","Type":"ContainerStarted","Data":"a0ceeaf3714ec8282d9900858a084918bc8cb333cd4fbb393449d4a40602aa29"} Jul 07 15:30:38.613759 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:38.613710 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" podStartSLOduration=1.613694494 podStartE2EDuration="1.613694494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:30:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:30:38.612440489 +0000 UTC m=+1787.428706549" watchObservedRunningTime="2026-07-07 15:30:38.613694494 +0000 UTC m=+1787.429960535" Jul 07 15:30:39.012521 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:39.012436 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:39.017468 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:39.017446 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:39.593854 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:39.593825 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:39.594954 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:39.594931 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:43.505418 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:43.505331 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb"] Jul 07 15:30:43.505846 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:43.505540 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" podUID="d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca" containerName="istio-proxy" containerID="cri-o://46e1df03aa8a70756ad8b500bc3fa3da4d8d3d8e276ba50b1a9c27246f8902f5" gracePeriod=30 Jul 07 15:30:48.641550 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.641515 2556 generic.go:356] "Generic (PLEG): container finished" podID="d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca" containerID="46e1df03aa8a70756ad8b500bc3fa3da4d8d3d8e276ba50b1a9c27246f8902f5" exitCode=0 Jul 07 15:30:48.641998 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.641639 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" event={"ID":"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca","Type":"ContainerDied","Data":"46e1df03aa8a70756ad8b500bc3fa3da4d8d3d8e276ba50b1a9c27246f8902f5"} Jul 07 15:30:48.752991 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.752968 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:48.837611 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.837581 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-workload-socket\") pod \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " Jul 07 15:30:48.837777 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.837625 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istiod-ca-cert\") pod \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " Jul 07 15:30:48.837777 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.837652 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-podinfo\") pod \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " Jul 07 15:30:48.837777 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.837685 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-credential-socket\") pod \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " Jul 07 15:30:48.837777 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.837705 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-token\") pod \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " Jul 07 15:30:48.837777 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.837732 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-envoy\") pod \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " Jul 07 15:30:48.837777 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.837749 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7b5l\" (UniqueName: \"kubernetes.io/projected/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-kube-api-access-d7b5l\") pod \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " Jul 07 15:30:48.838093 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.837820 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-data\") pod \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " Jul 07 15:30:48.838093 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.837850 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-workload-certs\") pod \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\" (UID: \"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca\") " Jul 07 15:30:48.838093 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.837855 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca" (UID: "d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:30:48.838093 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.838054 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca" (UID: "d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:30:48.838290 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.838105 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca" (UID: "d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:30:48.838290 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.838238 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-workload-socket\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:30:48.838290 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.838263 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istiod-ca-cert\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:30:48.838290 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.838278 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-credential-socket\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:30:48.838496 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.838321 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca" (UID: "d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:30:48.838496 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.838345 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-data" (OuterVolumeSpecName: "istio-data") pod "d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca" (UID: "d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:30:48.840036 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.840008 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca" (UID: "d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:30:48.840152 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.840032 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-kube-api-access-d7b5l" (OuterVolumeSpecName: "kube-api-access-d7b5l") pod "d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca" (UID: "d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca"). InnerVolumeSpecName "kube-api-access-d7b5l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:30:48.840152 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.840122 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-token" (OuterVolumeSpecName: "istio-token") pod "d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca" (UID: "d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:30:48.840247 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.840142 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca" (UID: "d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 15:30:48.938883 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.938827 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-podinfo\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:30:48.939072 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.938911 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-token\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:30:48.939072 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.938924 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-envoy\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:30:48.939072 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.938936 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7b5l\" (UniqueName: \"kubernetes.io/projected/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-kube-api-access-d7b5l\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:30:48.939072 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.938950 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-istio-data\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:30:48.939072 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:48.938962 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca-workload-certs\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:30:49.647853 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:49.647823 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" Jul 07 15:30:49.648314 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:49.647850 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb" event={"ID":"d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca","Type":"ContainerDied","Data":"a0ceeaf3714ec8282d9900858a084918bc8cb333cd4fbb393449d4a40602aa29"} Jul 07 15:30:49.648314 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:49.647935 2556 scope.go:117] "RemoveContainer" containerID="46e1df03aa8a70756ad8b500bc3fa3da4d8d3d8e276ba50b1a9c27246f8902f5" Jul 07 15:30:49.679163 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:49.679133 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb"] Jul 07 15:30:49.685381 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:49.685352 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-caa09bda-openshift-default-89c7b6767-5vwcb"] Jul 07 15:30:49.788010 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:49.787976 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca" path="/var/lib/kubelet/pods/d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca/volumes" Jul 07 15:30:51.731316 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:51.731287 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/1.log" Jul 07 15:30:51.734568 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:30:51.734540 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/1.log" Jul 07 15:32:26.858815 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:26.858779 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk"] Jul 07 15:32:26.859298 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:26.859185 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca" containerName="istio-proxy" Jul 07 15:32:26.859298 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:26.859197 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca" containerName="istio-proxy" Jul 07 15:32:26.859298 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:26.859261 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d39e44e1-4755-4b3a-b3e6-79c4a58ef5ca" containerName="istio-proxy" Jul 07 15:32:26.861514 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:26.861493 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:26.866283 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:26.866262 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-5a854080-openshift-default-dockercfg-zpp27\"" Jul 07 15:32:26.891918 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:26.891889 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk"] Jul 07 15:32:26.946487 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:26.946443 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e7bade5-8f20-4e35-a417-fed197f11344-istio-token\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:26.946657 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:26.946576 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-workload-socket\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:26.946657 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:26.946634 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-istio-envoy\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:26.946798 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:26.946662 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e7bade5-8f20-4e35-a417-fed197f11344-istio-podinfo\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:26.946798 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:26.946745 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddtl9\" (UniqueName: \"kubernetes.io/projected/1e7bade5-8f20-4e35-a417-fed197f11344-kube-api-access-ddtl9\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:26.946798 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:26.946783 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-credential-socket\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:26.946994 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:26.946826 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-istio-data\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:26.946994 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:26.946896 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e7bade5-8f20-4e35-a417-fed197f11344-istiod-ca-cert\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:26.946994 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:26.946948 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-workload-certs\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:27.048272 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.048237 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e7bade5-8f20-4e35-a417-fed197f11344-istio-token\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:27.048438 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.048295 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-workload-socket\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:27.048486 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.048461 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-istio-envoy\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:27.048529 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.048504 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e7bade5-8f20-4e35-a417-fed197f11344-istio-podinfo\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:27.048580 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.048564 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddtl9\" (UniqueName: \"kubernetes.io/projected/1e7bade5-8f20-4e35-a417-fed197f11344-kube-api-access-ddtl9\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:27.048635 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.048591 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-workload-socket\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:27.048635 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.048606 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-credential-socket\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:27.048744 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.048641 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-istio-data\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:27.048744 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.048660 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e7bade5-8f20-4e35-a417-fed197f11344-istiod-ca-cert\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:27.048744 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.048692 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-workload-certs\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:27.048938 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.048834 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-credential-socket\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:27.049019 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.048999 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-istio-data\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:27.049070 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.049054 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-workload-certs\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:27.049381 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.049361 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e7bade5-8f20-4e35-a417-fed197f11344-istiod-ca-cert\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:27.050593 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.050574 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-istio-envoy\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:27.051071 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.051052 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e7bade5-8f20-4e35-a417-fed197f11344-istio-podinfo\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:27.056938 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.056910 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e7bade5-8f20-4e35-a417-fed197f11344-istio-token\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:27.057063 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.057044 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddtl9\" (UniqueName: \"kubernetes.io/projected/1e7bade5-8f20-4e35-a417-fed197f11344-kube-api-access-ddtl9\") pod \"e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:27.172016 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.171939 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:27.319834 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.319810 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk"] Jul 07 15:32:27.322408 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:32:27.322380 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e7bade5_8f20_4e35_a417_fed197f11344.slice/crio-b16382fa4590cea11768cf54008b78128d507e43c00af2ce1faa7441f80e1917 WatchSource:0}: Error finding container b16382fa4590cea11768cf54008b78128d507e43c00af2ce1faa7441f80e1917: Status 404 returned error can't find the container with id b16382fa4590cea11768cf54008b78128d507e43c00af2ce1faa7441f80e1917 Jul 07 15:32:27.324760 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.324718 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:32:27.324836 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.324810 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:32:27.324891 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:27.324858 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:32:28.114028 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:28.113990 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" event={"ID":"1e7bade5-8f20-4e35-a417-fed197f11344","Type":"ContainerStarted","Data":"bcd2c026263afcc31c3add3e9fb848fb9e7cca9c21e546c35368ba6daad66885"} Jul 07 15:32:28.114422 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:28.114054 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" event={"ID":"1e7bade5-8f20-4e35-a417-fed197f11344","Type":"ContainerStarted","Data":"b16382fa4590cea11768cf54008b78128d507e43c00af2ce1faa7441f80e1917"} Jul 07 15:32:28.141194 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:28.141141 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" podStartSLOduration=2.141125285 podStartE2EDuration="2.141125285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:32:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:32:28.138850352 +0000 UTC m=+1896.955116425" watchObservedRunningTime="2026-07-07 15:32:28.141125285 +0000 UTC m=+1896.957391324" Jul 07 15:32:28.172224 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:28.172192 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:28.177275 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:28.177251 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:29.118747 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:29.118713 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:29.119772 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:29.119754 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:44.120783 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:44.120746 2556 patch_prober.go:28] interesting pod/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 07 15:32:44.121238 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:44.120814 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" podUID="1e7bade5-8f20-4e35-a417-fed197f11344" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 07 15:32:44.191454 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:44.191417 2556 generic.go:356] "Generic (PLEG): container finished" podID="1e7bade5-8f20-4e35-a417-fed197f11344" containerID="bcd2c026263afcc31c3add3e9fb848fb9e7cca9c21e546c35368ba6daad66885" exitCode=0 Jul 07 15:32:44.191611 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:44.191466 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" event={"ID":"1e7bade5-8f20-4e35-a417-fed197f11344","Type":"ContainerDied","Data":"bcd2c026263afcc31c3add3e9fb848fb9e7cca9c21e546c35368ba6daad66885"} Jul 07 15:32:44.191886 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:44.191856 2556 scope.go:117] "RemoveContainer" containerID="bcd2c026263afcc31c3add3e9fb848fb9e7cca9c21e546c35368ba6daad66885" Jul 07 15:32:44.192905 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:44.192855 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:32:44.192983 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:44.192964 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:32:44.193029 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:44.193013 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:32:45.199309 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:45.199270 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" event={"ID":"1e7bade5-8f20-4e35-a417-fed197f11344","Type":"ContainerStarted","Data":"40432e8e7399598bc93f536c0e198eacecf8d973b81fc1ee4046d07225001897"} Jul 07 15:32:46.172741 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:46.172702 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:46.173159 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:46.173061 2556 patch_prober.go:28] interesting pod/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 07 15:32:46.173338 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:46.173315 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" podUID="1e7bade5-8f20-4e35-a417-fed197f11344" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 07 15:32:47.172797 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:47.172749 2556 patch_prober.go:28] interesting pod/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 07 15:32:47.173341 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:47.172832 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" podUID="1e7bade5-8f20-4e35-a417-fed197f11344" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 07 15:32:48.173470 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:48.173403 2556 patch_prober.go:28] interesting pod/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 07 15:32:48.174645 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:48.174061 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" podUID="1e7bade5-8f20-4e35-a417-fed197f11344" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 07 15:32:48.797609 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:48.797521 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk"] Jul 07 15:32:48.797816 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:48.797794 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" podUID="1e7bade5-8f20-4e35-a417-fed197f11344" containerName="istio-proxy" containerID="cri-o://40432e8e7399598bc93f536c0e198eacecf8d973b81fc1ee4046d07225001897" gracePeriod=30 Jul 07 15:32:54.649552 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.649527 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:54.707767 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.707736 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e7bade5-8f20-4e35-a417-fed197f11344-istio-token\") pod \"1e7bade5-8f20-4e35-a417-fed197f11344\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " Jul 07 15:32:54.707767 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.707772 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-istio-envoy\") pod \"1e7bade5-8f20-4e35-a417-fed197f11344\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " Jul 07 15:32:54.708033 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.707815 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-workload-socket\") pod \"1e7bade5-8f20-4e35-a417-fed197f11344\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " Jul 07 15:32:54.708033 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.707850 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e7bade5-8f20-4e35-a417-fed197f11344-istio-podinfo\") pod \"1e7bade5-8f20-4e35-a417-fed197f11344\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " Jul 07 15:32:54.708033 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.707898 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddtl9\" (UniqueName: \"kubernetes.io/projected/1e7bade5-8f20-4e35-a417-fed197f11344-kube-api-access-ddtl9\") pod \"1e7bade5-8f20-4e35-a417-fed197f11344\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " Jul 07 15:32:54.708033 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.707927 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-credential-socket\") pod \"1e7bade5-8f20-4e35-a417-fed197f11344\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " Jul 07 15:32:54.708033 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.707946 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-istio-data\") pod \"1e7bade5-8f20-4e35-a417-fed197f11344\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " Jul 07 15:32:54.708297 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.708176 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1e7bade5-8f20-4e35-a417-fed197f11344" (UID: "1e7bade5-8f20-4e35-a417-fed197f11344"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:32:54.708297 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.708249 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e7bade5-8f20-4e35-a417-fed197f11344-istiod-ca-cert\") pod \"1e7bade5-8f20-4e35-a417-fed197f11344\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " Jul 07 15:32:54.708297 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.708293 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-workload-certs\") pod \"1e7bade5-8f20-4e35-a417-fed197f11344\" (UID: \"1e7bade5-8f20-4e35-a417-fed197f11344\") " Jul 07 15:32:54.708622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.708591 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e7bade5-8f20-4e35-a417-fed197f11344-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1e7bade5-8f20-4e35-a417-fed197f11344" (UID: "1e7bade5-8f20-4e35-a417-fed197f11344"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:32:54.708622 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.708613 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-workload-socket\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:32:54.708809 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.708785 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-istio-data" (OuterVolumeSpecName: "istio-data") pod "1e7bade5-8f20-4e35-a417-fed197f11344" (UID: "1e7bade5-8f20-4e35-a417-fed197f11344"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:32:54.708914 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.708809 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1e7bade5-8f20-4e35-a417-fed197f11344" (UID: "1e7bade5-8f20-4e35-a417-fed197f11344"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:32:54.708914 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.708850 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1e7bade5-8f20-4e35-a417-fed197f11344" (UID: "1e7bade5-8f20-4e35-a417-fed197f11344"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:32:54.710143 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.710115 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e7bade5-8f20-4e35-a417-fed197f11344-istio-token" (OuterVolumeSpecName: "istio-token") pod "1e7bade5-8f20-4e35-a417-fed197f11344" (UID: "1e7bade5-8f20-4e35-a417-fed197f11344"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:32:54.710254 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.710180 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e7bade5-8f20-4e35-a417-fed197f11344-kube-api-access-ddtl9" (OuterVolumeSpecName: "kube-api-access-ddtl9") pod "1e7bade5-8f20-4e35-a417-fed197f11344" (UID: "1e7bade5-8f20-4e35-a417-fed197f11344"). InnerVolumeSpecName "kube-api-access-ddtl9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:32:54.710353 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.710334 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1e7bade5-8f20-4e35-a417-fed197f11344-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1e7bade5-8f20-4e35-a417-fed197f11344" (UID: "1e7bade5-8f20-4e35-a417-fed197f11344"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 15:32:54.710434 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.710416 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1e7bade5-8f20-4e35-a417-fed197f11344" (UID: "1e7bade5-8f20-4e35-a417-fed197f11344"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:32:54.809941 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.809836 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e7bade5-8f20-4e35-a417-fed197f11344-istio-podinfo\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:32:54.809941 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.809906 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddtl9\" (UniqueName: \"kubernetes.io/projected/1e7bade5-8f20-4e35-a417-fed197f11344-kube-api-access-ddtl9\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:32:54.809941 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.809919 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-credential-socket\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:32:54.809941 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.809928 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-istio-data\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:32:54.809941 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.809938 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e7bade5-8f20-4e35-a417-fed197f11344-istiod-ca-cert\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:32:54.809941 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.809945 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-workload-certs\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:32:54.810246 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.809953 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e7bade5-8f20-4e35-a417-fed197f11344-istio-token\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:32:54.810246 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:54.809961 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e7bade5-8f20-4e35-a417-fed197f11344-istio-envoy\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:32:55.247036 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:55.246995 2556 generic.go:356] "Generic (PLEG): container finished" podID="1e7bade5-8f20-4e35-a417-fed197f11344" containerID="40432e8e7399598bc93f536c0e198eacecf8d973b81fc1ee4046d07225001897" exitCode=0 Jul 07 15:32:55.247220 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:55.247063 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" Jul 07 15:32:55.247220 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:55.247078 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" event={"ID":"1e7bade5-8f20-4e35-a417-fed197f11344","Type":"ContainerDied","Data":"40432e8e7399598bc93f536c0e198eacecf8d973b81fc1ee4046d07225001897"} Jul 07 15:32:55.247220 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:55.247137 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk" event={"ID":"1e7bade5-8f20-4e35-a417-fed197f11344","Type":"ContainerDied","Data":"b16382fa4590cea11768cf54008b78128d507e43c00af2ce1faa7441f80e1917"} Jul 07 15:32:55.247220 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:55.247160 2556 scope.go:117] "RemoveContainer" containerID="40432e8e7399598bc93f536c0e198eacecf8d973b81fc1ee4046d07225001897" Jul 07 15:32:55.256605 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:55.256581 2556 scope.go:117] "RemoveContainer" containerID="bcd2c026263afcc31c3add3e9fb848fb9e7cca9c21e546c35368ba6daad66885" Jul 07 15:32:55.264760 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:55.264740 2556 scope.go:117] "RemoveContainer" containerID="40432e8e7399598bc93f536c0e198eacecf8d973b81fc1ee4046d07225001897" Jul 07 15:32:55.265034 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:32:55.265014 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40432e8e7399598bc93f536c0e198eacecf8d973b81fc1ee4046d07225001897\": container with ID starting with 40432e8e7399598bc93f536c0e198eacecf8d973b81fc1ee4046d07225001897 not found: ID does not exist" containerID="40432e8e7399598bc93f536c0e198eacecf8d973b81fc1ee4046d07225001897" Jul 07 15:32:55.265101 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:55.265044 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40432e8e7399598bc93f536c0e198eacecf8d973b81fc1ee4046d07225001897"} err="failed to get container status \"40432e8e7399598bc93f536c0e198eacecf8d973b81fc1ee4046d07225001897\": rpc error: code = NotFound desc = could not find container \"40432e8e7399598bc93f536c0e198eacecf8d973b81fc1ee4046d07225001897\": container with ID starting with 40432e8e7399598bc93f536c0e198eacecf8d973b81fc1ee4046d07225001897 not found: ID does not exist" Jul 07 15:32:55.265101 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:55.265061 2556 scope.go:117] "RemoveContainer" containerID="bcd2c026263afcc31c3add3e9fb848fb9e7cca9c21e546c35368ba6daad66885" Jul 07 15:32:55.265267 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:32:55.265251 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcd2c026263afcc31c3add3e9fb848fb9e7cca9c21e546c35368ba6daad66885\": container with ID starting with bcd2c026263afcc31c3add3e9fb848fb9e7cca9c21e546c35368ba6daad66885 not found: ID does not exist" containerID="bcd2c026263afcc31c3add3e9fb848fb9e7cca9c21e546c35368ba6daad66885" Jul 07 15:32:55.265315 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:55.265272 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcd2c026263afcc31c3add3e9fb848fb9e7cca9c21e546c35368ba6daad66885"} err="failed to get container status \"bcd2c026263afcc31c3add3e9fb848fb9e7cca9c21e546c35368ba6daad66885\": rpc error: code = NotFound desc = could not find container \"bcd2c026263afcc31c3add3e9fb848fb9e7cca9c21e546c35368ba6daad66885\": container with ID starting with bcd2c026263afcc31c3add3e9fb848fb9e7cca9c21e546c35368ba6daad66885 not found: ID does not exist" Jul 07 15:32:55.272536 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:55.272513 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk"] Jul 07 15:32:55.277814 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:55.277793 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-5a854080-openshift-default-b8d74b895-qt4fk"] Jul 07 15:32:55.788615 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:32:55.788569 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e7bade5-8f20-4e35-a417-fed197f11344" path="/var/lib/kubelet/pods/1e7bade5-8f20-4e35-a417-fed197f11344/volumes" Jul 07 15:33:08.733709 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.733677 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s"] Jul 07 15:33:08.734249 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.734104 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e7bade5-8f20-4e35-a417-fed197f11344" containerName="istio-proxy" Jul 07 15:33:08.734249 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.734119 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e7bade5-8f20-4e35-a417-fed197f11344" containerName="istio-proxy" Jul 07 15:33:08.734249 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.734132 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e7bade5-8f20-4e35-a417-fed197f11344" containerName="istio-proxy" Jul 07 15:33:08.734249 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.734138 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e7bade5-8f20-4e35-a417-fed197f11344" containerName="istio-proxy" Jul 07 15:33:08.734249 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.734205 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e7bade5-8f20-4e35-a417-fed197f11344" containerName="istio-proxy" Jul 07 15:33:08.734249 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.734212 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e7bade5-8f20-4e35-a417-fed197f11344" containerName="istio-proxy" Jul 07 15:33:08.737449 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.737429 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.739909 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.739888 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-02505f19-openshift-default-dockercfg-65p4w\"" Jul 07 15:33:08.753473 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.753450 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s"] Jul 07 15:33:08.839527 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.839488 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-workload-certs\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.839527 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.839532 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-workload-socket\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.839791 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.839549 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0eead85-3b81-48f7-b109-d91f8a662c95-istiod-ca-cert\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.839791 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.839573 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-data\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.839791 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.839647 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-podinfo\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.839791 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.839689 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jlzz\" (UniqueName: \"kubernetes.io/projected/d0eead85-3b81-48f7-b109-d91f8a662c95-kube-api-access-2jlzz\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.839791 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.839749 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-token\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.840037 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.839814 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-credential-socket\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.840037 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.839863 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-envoy\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.941386 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.941347 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-workload-certs\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.941587 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.941401 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-workload-socket\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.941587 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.941430 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0eead85-3b81-48f7-b109-d91f8a662c95-istiod-ca-cert\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.941587 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.941464 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-data\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.941587 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.941506 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-podinfo\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.941587 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.941533 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jlzz\" (UniqueName: \"kubernetes.io/projected/d0eead85-3b81-48f7-b109-d91f8a662c95-kube-api-access-2jlzz\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.941587 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.941581 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-token\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.942076 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.941621 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-credential-socket\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.942076 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.941675 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-envoy\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.942076 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.941807 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-workload-certs\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.942676 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.942584 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-credential-socket\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.943276 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.943253 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0eead85-3b81-48f7-b109-d91f8a662c95-istiod-ca-cert\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.943521 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.943502 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-workload-socket\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.943751 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.943731 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-data\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.944280 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.944264 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-envoy\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.945444 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.945421 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-podinfo\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.950679 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.950661 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-token\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:08.950764 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:08.950685 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jlzz\" (UniqueName: \"kubernetes.io/projected/d0eead85-3b81-48f7-b109-d91f8a662c95-kube-api-access-2jlzz\") pod \"e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:09.047676 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:09.047595 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:09.187974 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:09.187765 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s"] Jul 07 15:33:09.190382 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:33:09.190348 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0eead85_3b81_48f7_b109_d91f8a662c95.slice/crio-19f9eb51314b7f31918508340d3cdb57f3bce3e386c35e63e3a9fe86362f0341 WatchSource:0}: Error finding container 19f9eb51314b7f31918508340d3cdb57f3bce3e386c35e63e3a9fe86362f0341: Status 404 returned error can't find the container with id 19f9eb51314b7f31918508340d3cdb57f3bce3e386c35e63e3a9fe86362f0341 Jul 07 15:33:09.192611 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:09.192574 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:33:09.192676 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:09.192659 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:33:09.192725 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:09.192710 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:33:09.317142 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:09.317040 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" event={"ID":"d0eead85-3b81-48f7-b109-d91f8a662c95","Type":"ContainerStarted","Data":"86edb2d8964038adeafbdb9040409eed60b27e635dac773bf1b14a9aae7b8c55"} Jul 07 15:33:09.317295 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:09.317156 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" event={"ID":"d0eead85-3b81-48f7-b109-d91f8a662c95","Type":"ContainerStarted","Data":"19f9eb51314b7f31918508340d3cdb57f3bce3e386c35e63e3a9fe86362f0341"} Jul 07 15:33:09.337021 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:09.336970 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" podStartSLOduration=1.336955295 podStartE2EDuration="1.336955295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:33:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:33:09.335114378 +0000 UTC m=+1938.151380431" watchObservedRunningTime="2026-07-07 15:33:09.336955295 +0000 UTC m=+1938.153221335" Jul 07 15:33:10.048611 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:10.048581 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:10.053513 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:10.053490 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:10.322269 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:10.322178 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:10.323165 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:10.323142 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:25.323891 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:25.323825 2556 patch_prober.go:28] interesting pod/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 07 15:33:25.324291 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:25.323911 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" podUID="d0eead85-3b81-48f7-b109-d91f8a662c95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 07 15:33:25.403603 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:25.403569 2556 generic.go:356] "Generic (PLEG): container finished" podID="d0eead85-3b81-48f7-b109-d91f8a662c95" containerID="86edb2d8964038adeafbdb9040409eed60b27e635dac773bf1b14a9aae7b8c55" exitCode=0 Jul 07 15:33:25.403777 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:25.403629 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" event={"ID":"d0eead85-3b81-48f7-b109-d91f8a662c95","Type":"ContainerDied","Data":"86edb2d8964038adeafbdb9040409eed60b27e635dac773bf1b14a9aae7b8c55"} Jul 07 15:33:25.404056 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:25.404036 2556 scope.go:117] "RemoveContainer" containerID="86edb2d8964038adeafbdb9040409eed60b27e635dac773bf1b14a9aae7b8c55" Jul 07 15:33:25.405074 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:25.405043 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:33:25.405182 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:25.405113 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:33:25.405182 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:25.405145 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:33:26.410487 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:26.410452 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" event={"ID":"d0eead85-3b81-48f7-b109-d91f8a662c95","Type":"ContainerStarted","Data":"dfa34805f640ca523582f7c9b3544ccc84b4f9c8b56e0c7f10c9ad82088ebaa3"} Jul 07 15:33:27.047878 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:27.047847 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:27.048117 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:27.048097 2556 patch_prober.go:28] interesting pod/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 07 15:33:27.048170 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:27.048151 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" podUID="d0eead85-3b81-48f7-b109-d91f8a662c95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 07 15:33:28.048976 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.048934 2556 patch_prober.go:28] interesting pod/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 07 15:33:28.049392 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.049008 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" podUID="d0eead85-3b81-48f7-b109-d91f8a662c95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 07 15:33:28.698488 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.698443 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96"] Jul 07 15:33:28.701737 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.701714 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:28.705035 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.705010 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-02505f19\"/\"openshift-service-ca.crt\"" Jul 07 15:33:28.705035 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.705030 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-02505f19\"/\"kube-root-ca.crt\"" Jul 07 15:33:28.705224 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.705017 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-02505f19\"/\"default-dockercfg-g5fh9\"" Jul 07 15:33:28.705224 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.705090 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-02505f19\"/\"e2e-lifecycle-model-02505f19-kserve-self-signed-certs\"" Jul 07 15:33:28.713540 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.713515 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96"] Jul 07 15:33:28.736160 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.736123 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-tmp-dir\") pod \"e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:28.736387 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.736218 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-home\") pod \"e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:28.736387 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.736242 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-dshm\") pod \"e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:28.736387 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.736267 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9ptc\" (UniqueName: \"kubernetes.io/projected/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-kube-api-access-x9ptc\") pod \"e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:28.736537 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.736373 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-model-cache\") pod \"e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:28.736537 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.736414 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-tls-certs\") pod \"e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:28.837753 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.837705 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-home\") pod \"e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:28.837753 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.837752 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-dshm\") pod \"e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:28.838054 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.837785 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9ptc\" (UniqueName: \"kubernetes.io/projected/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-kube-api-access-x9ptc\") pod \"e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:28.838054 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.837820 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-model-cache\") pod \"e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:28.838054 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.837842 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-tls-certs\") pod \"e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:28.838054 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.837946 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-tmp-dir\") pod \"e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:28.838265 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.838218 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-home\") pod \"e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:28.838265 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.838242 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-model-cache\") pod \"e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:28.838366 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.838324 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-tmp-dir\") pod \"e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:28.840175 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.840152 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-dshm\") pod \"e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:28.840435 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.840416 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-tls-certs\") pod \"e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:28.846401 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:28.846374 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9ptc\" (UniqueName: \"kubernetes.io/projected/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-kube-api-access-x9ptc\") pod \"e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:29.018919 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:29.018796 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:29.048360 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:29.048322 2556 patch_prober.go:28] interesting pod/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 07 15:33:29.048512 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:29.048401 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" podUID="d0eead85-3b81-48f7-b109-d91f8a662c95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 07 15:33:29.180964 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:29.175485 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96"] Jul 07 15:33:29.192237 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:33:29.192194 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb83e6bd2_e25d_4e29_8b40_223b07d9cd55.slice/crio-53aa59e5bfe5a51b3f2595c66b4b0377d110cbe9ccb114afc3fadb01e2fb99f4 WatchSource:0}: Error finding container 53aa59e5bfe5a51b3f2595c66b4b0377d110cbe9ccb114afc3fadb01e2fb99f4: Status 404 returned error can't find the container with id 53aa59e5bfe5a51b3f2595c66b4b0377d110cbe9ccb114afc3fadb01e2fb99f4 Jul 07 15:33:29.427388 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:29.427352 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" event={"ID":"b83e6bd2-e25d-4e29-8b40-223b07d9cd55","Type":"ContainerStarted","Data":"9e46f85c2f2802702c03ff5bd1491f925fdf0a6c4eced5fa539bfdfb1382a3ee"} Jul 07 15:33:29.427388 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:29.427393 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" event={"ID":"b83e6bd2-e25d-4e29-8b40-223b07d9cd55","Type":"ContainerStarted","Data":"53aa59e5bfe5a51b3f2595c66b4b0377d110cbe9ccb114afc3fadb01e2fb99f4"} Jul 07 15:33:29.447185 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:29.447133 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" podStartSLOduration=1.447116402 podStartE2EDuration="1.447116402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:33:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:33:29.444522631 +0000 UTC m=+1958.260788670" watchObservedRunningTime="2026-07-07 15:33:29.447116402 +0000 UTC m=+1958.263382441" Jul 07 15:33:30.048787 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:30.048748 2556 patch_prober.go:28] interesting pod/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 07 15:33:30.049006 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:30.048813 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" podUID="d0eead85-3b81-48f7-b109-d91f8a662c95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 07 15:33:31.052222 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:31.052193 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:31.052615 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:31.052529 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:31.053200 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:31.053182 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:33:39.019373 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:39.019334 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:39.019821 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:39.019601 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:39.031965 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:39.031938 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:33:39.487732 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:33:39.487703 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:34:04.620190 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:04.620153 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96"] Jul 07 15:34:19.641068 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:19.641016 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" podUID="b83e6bd2-e25d-4e29-8b40-223b07d9cd55" containerName="main" containerID="cri-o://9e46f85c2f2802702c03ff5bd1491f925fdf0a6c4eced5fa539bfdfb1382a3ee" gracePeriod=45 Jul 07 15:34:19.886005 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:19.885979 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:34:20.030700 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.030608 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-tls-certs\") pod \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " Jul 07 15:34:20.030700 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.030663 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9ptc\" (UniqueName: \"kubernetes.io/projected/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-kube-api-access-x9ptc\") pod \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " Jul 07 15:34:20.030889 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.030784 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-model-cache\") pod \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " Jul 07 15:34:20.030889 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.030841 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-dshm\") pod \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " Jul 07 15:34:20.031044 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.030910 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-home\") pod \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " Jul 07 15:34:20.031044 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.030953 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-tmp-dir\") pod \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\" (UID: \"b83e6bd2-e25d-4e29-8b40-223b07d9cd55\") " Jul 07 15:34:20.031152 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.031032 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-model-cache" (OuterVolumeSpecName: "model-cache") pod "b83e6bd2-e25d-4e29-8b40-223b07d9cd55" (UID: "b83e6bd2-e25d-4e29-8b40-223b07d9cd55"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:34:20.031276 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.031192 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-home" (OuterVolumeSpecName: "home") pod "b83e6bd2-e25d-4e29-8b40-223b07d9cd55" (UID: "b83e6bd2-e25d-4e29-8b40-223b07d9cd55"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:34:20.031276 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.031255 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b83e6bd2-e25d-4e29-8b40-223b07d9cd55" (UID: "b83e6bd2-e25d-4e29-8b40-223b07d9cd55"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:34:20.031450 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.031361 2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-tmp-dir\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:34:20.031450 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.031376 2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-model-cache\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:34:20.031450 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.031390 2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-home\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:34:20.033099 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.033073 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-dshm" (OuterVolumeSpecName: "dshm") pod "b83e6bd2-e25d-4e29-8b40-223b07d9cd55" (UID: "b83e6bd2-e25d-4e29-8b40-223b07d9cd55"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:34:20.033208 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.033146 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-kube-api-access-x9ptc" (OuterVolumeSpecName: "kube-api-access-x9ptc") pod "b83e6bd2-e25d-4e29-8b40-223b07d9cd55" (UID: "b83e6bd2-e25d-4e29-8b40-223b07d9cd55"). InnerVolumeSpecName "kube-api-access-x9ptc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:34:20.033208 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.033163 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b83e6bd2-e25d-4e29-8b40-223b07d9cd55" (UID: "b83e6bd2-e25d-4e29-8b40-223b07d9cd55"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 15:34:20.132439 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.132394 2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-dshm\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:34:20.132439 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.132436 2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-tls-certs\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:34:20.132659 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.132454 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9ptc\" (UniqueName: \"kubernetes.io/projected/b83e6bd2-e25d-4e29-8b40-223b07d9cd55-kube-api-access-x9ptc\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:34:20.671999 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.671963 2556 generic.go:356] "Generic (PLEG): container finished" podID="b83e6bd2-e25d-4e29-8b40-223b07d9cd55" containerID="9e46f85c2f2802702c03ff5bd1491f925fdf0a6c4eced5fa539bfdfb1382a3ee" exitCode=0 Jul 07 15:34:20.672504 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.672013 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" Jul 07 15:34:20.672504 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.672052 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" event={"ID":"b83e6bd2-e25d-4e29-8b40-223b07d9cd55","Type":"ContainerDied","Data":"9e46f85c2f2802702c03ff5bd1491f925fdf0a6c4eced5fa539bfdfb1382a3ee"} Jul 07 15:34:20.672504 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.672108 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96" event={"ID":"b83e6bd2-e25d-4e29-8b40-223b07d9cd55","Type":"ContainerDied","Data":"53aa59e5bfe5a51b3f2595c66b4b0377d110cbe9ccb114afc3fadb01e2fb99f4"} Jul 07 15:34:20.672504 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.672134 2556 scope.go:117] "RemoveContainer" containerID="9e46f85c2f2802702c03ff5bd1491f925fdf0a6c4eced5fa539bfdfb1382a3ee" Jul 07 15:34:20.681765 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.681747 2556 scope.go:117] "RemoveContainer" containerID="9e46f85c2f2802702c03ff5bd1491f925fdf0a6c4eced5fa539bfdfb1382a3ee" Jul 07 15:34:20.682091 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:34:20.682071 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e46f85c2f2802702c03ff5bd1491f925fdf0a6c4eced5fa539bfdfb1382a3ee\": container with ID starting with 9e46f85c2f2802702c03ff5bd1491f925fdf0a6c4eced5fa539bfdfb1382a3ee not found: ID does not exist" containerID="9e46f85c2f2802702c03ff5bd1491f925fdf0a6c4eced5fa539bfdfb1382a3ee" Jul 07 15:34:20.682164 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.682105 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e46f85c2f2802702c03ff5bd1491f925fdf0a6c4eced5fa539bfdfb1382a3ee"} err="failed to get container status \"9e46f85c2f2802702c03ff5bd1491f925fdf0a6c4eced5fa539bfdfb1382a3ee\": rpc error: code = NotFound desc = could not find container \"9e46f85c2f2802702c03ff5bd1491f925fdf0a6c4eced5fa539bfdfb1382a3ee\": container with ID starting with 9e46f85c2f2802702c03ff5bd1491f925fdf0a6c4eced5fa539bfdfb1382a3ee not found: ID does not exist" Jul 07 15:34:20.695182 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.695149 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96"] Jul 07 15:34:20.699278 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:20.699239 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-02505f19/e2e-lifecycle-model-02505f19-kserve-769468f8d7-ndg96"] Jul 07 15:34:21.788663 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:21.788629 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b83e6bd2-e25d-4e29-8b40-223b07d9cd55" path="/var/lib/kubelet/pods/b83e6bd2-e25d-4e29-8b40-223b07d9cd55/volumes" Jul 07 15:34:47.425025 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:47.424985 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s"] Jul 07 15:34:47.425534 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:47.425254 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" podUID="d0eead85-3b81-48f7-b109-d91f8a662c95" containerName="istio-proxy" containerID="cri-o://dfa34805f640ca523582f7c9b3544ccc84b4f9c8b56e0c7f10c9ad82088ebaa3" gracePeriod=30 Jul 07 15:34:52.679624 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.679599 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:34:52.747468 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.747380 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0eead85-3b81-48f7-b109-d91f8a662c95-istiod-ca-cert\") pod \"d0eead85-3b81-48f7-b109-d91f8a662c95\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " Jul 07 15:34:52.747468 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.747435 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-token\") pod \"d0eead85-3b81-48f7-b109-d91f8a662c95\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " Jul 07 15:34:52.747772 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.747476 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-credential-socket\") pod \"d0eead85-3b81-48f7-b109-d91f8a662c95\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " Jul 07 15:34:52.747772 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.747507 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-workload-socket\") pod \"d0eead85-3b81-48f7-b109-d91f8a662c95\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " Jul 07 15:34:52.747772 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.747542 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-envoy\") pod \"d0eead85-3b81-48f7-b109-d91f8a662c95\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " Jul 07 15:34:52.747772 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.747612 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-workload-certs\") pod \"d0eead85-3b81-48f7-b109-d91f8a662c95\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " Jul 07 15:34:52.747772 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.747699 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jlzz\" (UniqueName: \"kubernetes.io/projected/d0eead85-3b81-48f7-b109-d91f8a662c95-kube-api-access-2jlzz\") pod \"d0eead85-3b81-48f7-b109-d91f8a662c95\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " Jul 07 15:34:52.747772 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.747729 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-podinfo\") pod \"d0eead85-3b81-48f7-b109-d91f8a662c95\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " Jul 07 15:34:52.747772 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.747768 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-data\") pod \"d0eead85-3b81-48f7-b109-d91f8a662c95\" (UID: \"d0eead85-3b81-48f7-b109-d91f8a662c95\") " Jul 07 15:34:52.748138 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.747835 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d0eead85-3b81-48f7-b109-d91f8a662c95" (UID: "d0eead85-3b81-48f7-b109-d91f8a662c95"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:34:52.748138 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.747845 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d0eead85-3b81-48f7-b109-d91f8a662c95" (UID: "d0eead85-3b81-48f7-b109-d91f8a662c95"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:34:52.748138 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.747861 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0eead85-3b81-48f7-b109-d91f8a662c95-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d0eead85-3b81-48f7-b109-d91f8a662c95" (UID: "d0eead85-3b81-48f7-b109-d91f8a662c95"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:34:52.748138 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.748013 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d0eead85-3b81-48f7-b109-d91f8a662c95" (UID: "d0eead85-3b81-48f7-b109-d91f8a662c95"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:34:52.748318 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.748161 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-workload-certs\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:34:52.748318 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.748183 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d0eead85-3b81-48f7-b109-d91f8a662c95-istiod-ca-cert\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:34:52.748318 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.748200 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-credential-socket\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:34:52.748318 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.748217 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-workload-socket\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:34:52.748980 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.748949 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-data" (OuterVolumeSpecName: "istio-data") pod "d0eead85-3b81-48f7-b109-d91f8a662c95" (UID: "d0eead85-3b81-48f7-b109-d91f8a662c95"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:34:52.749830 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.749800 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d0eead85-3b81-48f7-b109-d91f8a662c95" (UID: "d0eead85-3b81-48f7-b109-d91f8a662c95"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:34:52.750035 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.750007 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d0eead85-3b81-48f7-b109-d91f8a662c95" (UID: "d0eead85-3b81-48f7-b109-d91f8a662c95"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 15:34:52.750131 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.750036 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-token" (OuterVolumeSpecName: "istio-token") pod "d0eead85-3b81-48f7-b109-d91f8a662c95" (UID: "d0eead85-3b81-48f7-b109-d91f8a662c95"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:34:52.750131 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.750087 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0eead85-3b81-48f7-b109-d91f8a662c95-kube-api-access-2jlzz" (OuterVolumeSpecName: "kube-api-access-2jlzz") pod "d0eead85-3b81-48f7-b109-d91f8a662c95" (UID: "d0eead85-3b81-48f7-b109-d91f8a662c95"). InnerVolumeSpecName "kube-api-access-2jlzz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:34:52.826017 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.825982 2556 generic.go:356] "Generic (PLEG): container finished" podID="d0eead85-3b81-48f7-b109-d91f8a662c95" containerID="dfa34805f640ca523582f7c9b3544ccc84b4f9c8b56e0c7f10c9ad82088ebaa3" exitCode=0 Jul 07 15:34:52.826204 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.826046 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" Jul 07 15:34:52.826204 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.826066 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" event={"ID":"d0eead85-3b81-48f7-b109-d91f8a662c95","Type":"ContainerDied","Data":"dfa34805f640ca523582f7c9b3544ccc84b4f9c8b56e0c7f10c9ad82088ebaa3"} Jul 07 15:34:52.826204 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.826121 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s" event={"ID":"d0eead85-3b81-48f7-b109-d91f8a662c95","Type":"ContainerDied","Data":"19f9eb51314b7f31918508340d3cdb57f3bce3e386c35e63e3a9fe86362f0341"} Jul 07 15:34:52.826204 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.826143 2556 scope.go:117] "RemoveContainer" containerID="dfa34805f640ca523582f7c9b3544ccc84b4f9c8b56e0c7f10c9ad82088ebaa3" Jul 07 15:34:52.835112 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.835096 2556 scope.go:117] "RemoveContainer" containerID="86edb2d8964038adeafbdb9040409eed60b27e635dac773bf1b14a9aae7b8c55" Jul 07 15:34:52.842674 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.842658 2556 scope.go:117] "RemoveContainer" containerID="dfa34805f640ca523582f7c9b3544ccc84b4f9c8b56e0c7f10c9ad82088ebaa3" Jul 07 15:34:52.842950 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:34:52.842934 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfa34805f640ca523582f7c9b3544ccc84b4f9c8b56e0c7f10c9ad82088ebaa3\": container with ID starting with dfa34805f640ca523582f7c9b3544ccc84b4f9c8b56e0c7f10c9ad82088ebaa3 not found: ID does not exist" containerID="dfa34805f640ca523582f7c9b3544ccc84b4f9c8b56e0c7f10c9ad82088ebaa3" Jul 07 15:34:52.843002 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.842958 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfa34805f640ca523582f7c9b3544ccc84b4f9c8b56e0c7f10c9ad82088ebaa3"} err="failed to get container status \"dfa34805f640ca523582f7c9b3544ccc84b4f9c8b56e0c7f10c9ad82088ebaa3\": rpc error: code = NotFound desc = could not find container \"dfa34805f640ca523582f7c9b3544ccc84b4f9c8b56e0c7f10c9ad82088ebaa3\": container with ID starting with dfa34805f640ca523582f7c9b3544ccc84b4f9c8b56e0c7f10c9ad82088ebaa3 not found: ID does not exist" Jul 07 15:34:52.843002 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.842974 2556 scope.go:117] "RemoveContainer" containerID="86edb2d8964038adeafbdb9040409eed60b27e635dac773bf1b14a9aae7b8c55" Jul 07 15:34:52.843216 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:34:52.843201 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86edb2d8964038adeafbdb9040409eed60b27e635dac773bf1b14a9aae7b8c55\": container with ID starting with 86edb2d8964038adeafbdb9040409eed60b27e635dac773bf1b14a9aae7b8c55 not found: ID does not exist" containerID="86edb2d8964038adeafbdb9040409eed60b27e635dac773bf1b14a9aae7b8c55" Jul 07 15:34:52.843274 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.843219 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86edb2d8964038adeafbdb9040409eed60b27e635dac773bf1b14a9aae7b8c55"} err="failed to get container status \"86edb2d8964038adeafbdb9040409eed60b27e635dac773bf1b14a9aae7b8c55\": rpc error: code = NotFound desc = could not find container \"86edb2d8964038adeafbdb9040409eed60b27e635dac773bf1b14a9aae7b8c55\": container with ID starting with 86edb2d8964038adeafbdb9040409eed60b27e635dac773bf1b14a9aae7b8c55 not found: ID does not exist" Jul 07 15:34:52.848472 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.848449 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s"] Jul 07 15:34:52.848689 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.848671 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-envoy\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:34:52.848768 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.848691 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jlzz\" (UniqueName: \"kubernetes.io/projected/d0eead85-3b81-48f7-b109-d91f8a662c95-kube-api-access-2jlzz\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:34:52.848768 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.848702 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-podinfo\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:34:52.848768 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.848710 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-data\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:34:52.848768 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.848720 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0eead85-3b81-48f7-b109-d91f8a662c95-istio-token\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:34:52.858885 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:52.858838 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-02505f19-openshift-default-688d4bccdc-s6m4s"] Jul 07 15:34:53.789497 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:34:53.789466 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0eead85-3b81-48f7-b109-d91f8a662c95" path="/var/lib/kubelet/pods/d0eead85-3b81-48f7-b109-d91f8a662c95/volumes" Jul 07 15:35:42.697266 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.697235 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g"] Jul 07 15:35:42.697675 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.697598 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0eead85-3b81-48f7-b109-d91f8a662c95" containerName="istio-proxy" Jul 07 15:35:42.697675 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.697610 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0eead85-3b81-48f7-b109-d91f8a662c95" containerName="istio-proxy" Jul 07 15:35:42.697675 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.697622 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0eead85-3b81-48f7-b109-d91f8a662c95" containerName="istio-proxy" Jul 07 15:35:42.697675 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.697629 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0eead85-3b81-48f7-b109-d91f8a662c95" containerName="istio-proxy" Jul 07 15:35:42.697675 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.697642 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b83e6bd2-e25d-4e29-8b40-223b07d9cd55" containerName="main" Jul 07 15:35:42.697675 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.697647 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="b83e6bd2-e25d-4e29-8b40-223b07d9cd55" containerName="main" Jul 07 15:35:42.697860 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.697707 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0eead85-3b81-48f7-b109-d91f8a662c95" containerName="istio-proxy" Jul 07 15:35:42.697860 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.697718 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="b83e6bd2-e25d-4e29-8b40-223b07d9cd55" containerName="main" Jul 07 15:35:42.700693 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.700671 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.706511 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.706489 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-6c4fac-openshift-default-dockercfg-wx6pw\"" Jul 07 15:35:42.716883 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.716848 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g"] Jul 07 15:35:42.816373 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.816341 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-token\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.816563 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.816384 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-credential-socket\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.816563 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.816418 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-data\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.816563 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.816499 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5svqh\" (UniqueName: \"kubernetes.io/projected/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-kube-api-access-5svqh\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.816734 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.816562 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istiod-ca-cert\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.816734 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.816597 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-podinfo\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.816734 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.816629 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-envoy\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.816734 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.816659 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-workload-certs\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.816734 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.816695 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-workload-socket\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.917527 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.917486 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-workload-socket\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.917725 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.917554 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-token\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.917923 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.917861 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-credential-socket\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.918029 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.917963 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-workload-socket\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.918029 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.917990 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-data\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.918155 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.918082 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5svqh\" (UniqueName: \"kubernetes.io/projected/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-kube-api-access-5svqh\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.918155 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.918110 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-credential-socket\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.918262 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.918154 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istiod-ca-cert\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.918262 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.918187 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-podinfo\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.918262 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.918232 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-envoy\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.918404 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.918263 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-data\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.918404 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.918271 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-workload-certs\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.918530 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.918512 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-workload-certs\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.918778 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.918756 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istiod-ca-cert\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.920457 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.920434 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-envoy\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.920677 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.920656 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-podinfo\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.925990 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.925965 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-token\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:42.926175 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:42.926154 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5svqh\" (UniqueName: \"kubernetes.io/projected/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-kube-api-access-5svqh\") pod \"e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:43.011131 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:43.011036 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:43.156025 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:43.156000 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g"] Jul 07 15:35:43.158268 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:35:43.158227 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3a2f7b9_250a_4d35_a7b2_13fa56359b6c.slice/crio-698133965c625b74a8e05801fafb07b0f6c9dce70f962493ad156a999242f15e WatchSource:0}: Error finding container 698133965c625b74a8e05801fafb07b0f6c9dce70f962493ad156a999242f15e: Status 404 returned error can't find the container with id 698133965c625b74a8e05801fafb07b0f6c9dce70f962493ad156a999242f15e Jul 07 15:35:43.160581 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:43.160547 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:35:43.160680 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:43.160616 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:35:43.160680 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:43.160645 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:35:44.075513 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:44.075478 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" event={"ID":"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c","Type":"ContainerStarted","Data":"c538e8985611e73702ada440c58593ff8931283fd5599ef5070707389cc06646"} Jul 07 15:35:44.075976 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:44.075544 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" event={"ID":"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c","Type":"ContainerStarted","Data":"698133965c625b74a8e05801fafb07b0f6c9dce70f962493ad156a999242f15e"} Jul 07 15:35:44.102461 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:44.102402 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" podStartSLOduration=2.102384474 podStartE2EDuration="2.102384474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:35:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:35:44.094737781 +0000 UTC m=+2092.911003822" watchObservedRunningTime="2026-07-07 15:35:44.102384474 +0000 UTC m=+2092.918650513" Jul 07 15:35:45.012077 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:45.012036 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:45.017110 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:45.017082 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:45.080406 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:45.080372 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:45.081345 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:45.081325 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:35:51.328035 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.327998 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb"] Jul 07 15:35:51.328495 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.328479 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0eead85-3b81-48f7-b109-d91f8a662c95" containerName="istio-proxy" Jul 07 15:35:51.332439 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.332416 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.334969 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.334948 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-c5eea1-openshift-default-dockercfg-nkzcp\"" Jul 07 15:35:51.350966 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.350943 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb"] Jul 07 15:35:51.502857 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.502822 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkc66\" (UniqueName: \"kubernetes.io/projected/75273620-36d8-4049-8ab9-db420f0fad8a-kube-api-access-hkc66\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.503051 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.502894 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-istio-envoy\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.503051 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.502989 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75273620-36d8-4049-8ab9-db420f0fad8a-istiod-ca-cert\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.503149 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.503052 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75273620-36d8-4049-8ab9-db420f0fad8a-istio-podinfo\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.503149 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.503092 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-workload-certs\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.503149 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.503132 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75273620-36d8-4049-8ab9-db420f0fad8a-istio-token\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.503274 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.503157 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-workload-socket\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.503274 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.503254 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-credential-socket\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.503341 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.503304 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-istio-data\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.604102 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.604015 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-credential-socket\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.604102 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.604066 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-istio-data\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.604102 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.604092 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkc66\" (UniqueName: \"kubernetes.io/projected/75273620-36d8-4049-8ab9-db420f0fad8a-kube-api-access-hkc66\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.604367 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.604120 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-istio-envoy\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.604367 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.604142 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75273620-36d8-4049-8ab9-db420f0fad8a-istiod-ca-cert\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.604367 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.604169 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75273620-36d8-4049-8ab9-db420f0fad8a-istio-podinfo\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.604367 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.604189 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-workload-certs\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.604367 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.604210 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75273620-36d8-4049-8ab9-db420f0fad8a-istio-token\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.604367 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.604289 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-workload-socket\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.604677 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.604439 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-istio-data\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.604677 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.604519 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-workload-certs\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.604677 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.604604 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-workload-socket\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.604677 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.604665 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-credential-socket\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.604845 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.604824 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75273620-36d8-4049-8ab9-db420f0fad8a-istiod-ca-cert\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.606574 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.606553 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-istio-envoy\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.606784 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.606766 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75273620-36d8-4049-8ab9-db420f0fad8a-istio-podinfo\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.612104 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.612082 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75273620-36d8-4049-8ab9-db420f0fad8a-istio-token\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.612443 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.612424 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkc66\" (UniqueName: \"kubernetes.io/projected/75273620-36d8-4049-8ab9-db420f0fad8a-kube-api-access-hkc66\") pod \"e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.644918 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.644891 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-c5eea1-openshift-default-dockercfg-nkzcp\"" Jul 07 15:35:51.653167 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.653146 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:51.774389 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.774354 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/1.log" Jul 07 15:35:51.776770 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.776723 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/1.log" Jul 07 15:35:51.806722 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.806698 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb"] Jul 07 15:35:51.808730 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:35:51.808701 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75273620_36d8_4049_8ab9_db420f0fad8a.slice/crio-12af435b387c949856e4cb546e6e1b0ce6791af72602a4662012fe3423214131 WatchSource:0}: Error finding container 12af435b387c949856e4cb546e6e1b0ce6791af72602a4662012fe3423214131: Status 404 returned error can't find the container with id 12af435b387c949856e4cb546e6e1b0ce6791af72602a4662012fe3423214131 Jul 07 15:35:51.811253 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.811209 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:35:51.811379 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.811284 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:35:51.811379 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:51.811313 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:35:52.114376 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:52.114282 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" event={"ID":"75273620-36d8-4049-8ab9-db420f0fad8a","Type":"ContainerStarted","Data":"ac82c5215a1c06e91f875e02f8baced6e3ec7c5a2782268303e9980a0b8639b2"} Jul 07 15:35:52.114376 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:52.114348 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" event={"ID":"75273620-36d8-4049-8ab9-db420f0fad8a","Type":"ContainerStarted","Data":"12af435b387c949856e4cb546e6e1b0ce6791af72602a4662012fe3423214131"} Jul 07 15:35:52.136543 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:52.136478 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" podStartSLOduration=1.136458024 podStartE2EDuration="1.136458024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:35:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:35:52.136116365 +0000 UTC m=+2100.952382405" watchObservedRunningTime="2026-07-07 15:35:52.136458024 +0000 UTC m=+2100.952724063" Jul 07 15:35:52.653771 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:52.653728 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:52.659509 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:52.659484 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:53.119279 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:53.119241 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:35:53.120176 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:35:53.120156 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:36:01.121060 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:01.121011 2556 patch_prober.go:28] interesting pod/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 15:36:01.121442 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:01.121107 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" podUID="f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 15:36:03.172915 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:03.172859 2556 generic.go:356] "Generic (PLEG): container finished" podID="f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" containerID="c538e8985611e73702ada440c58593ff8931283fd5599ef5070707389cc06646" exitCode=0 Jul 07 15:36:03.173502 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:03.172904 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" event={"ID":"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c","Type":"ContainerDied","Data":"c538e8985611e73702ada440c58593ff8931283fd5599ef5070707389cc06646"} Jul 07 15:36:03.174176 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:03.174157 2556 scope.go:117] "RemoveContainer" containerID="c538e8985611e73702ada440c58593ff8931283fd5599ef5070707389cc06646" Jul 07 15:36:03.179671 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:03.179616 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:36:03.179966 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:03.179907 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:36:03.180165 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:03.180120 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:36:04.183905 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:04.183848 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" event={"ID":"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c","Type":"ContainerStarted","Data":"c6ce9a5a1073f99dcb2ff54c8cbace525bdb58a8d3a523203c0565b1348cd6af"} Jul 07 15:36:05.011649 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:05.011610 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:36:05.011938 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:05.011912 2556 patch_prober.go:28] interesting pod/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 07 15:36:05.012017 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:05.011993 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" podUID="f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 07 15:36:06.012399 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:06.012357 2556 patch_prober.go:28] interesting pod/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 07 15:36:06.012806 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:06.012428 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" podUID="f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 07 15:36:07.011862 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:07.011823 2556 patch_prober.go:28] interesting pod/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 07 15:36:07.012084 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:07.011914 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" podUID="f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 07 15:36:08.011721 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:08.011677 2556 patch_prober.go:28] interesting pod/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 07 15:36:08.012224 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:08.011769 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" podUID="f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 07 15:36:08.407566 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:08.407301 2556 patch_prober.go:28] interesting pod/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": read tcp 10.134.0.2:41986->10.134.0.37:15021: read: connection reset by peer" start-of-body= Jul 07 15:36:08.407566 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:08.407366 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" podUID="75273620-36d8-4049-8ab9-db420f0fad8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": read tcp 10.134.0.2:41986->10.134.0.37:15021: read: connection reset by peer" Jul 07 15:36:09.015424 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:09.015394 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:36:09.015810 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:09.015670 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:36:09.016341 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:09.016323 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:36:09.214879 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:09.214830 2556 generic.go:356] "Generic (PLEG): container finished" podID="75273620-36d8-4049-8ab9-db420f0fad8a" containerID="ac82c5215a1c06e91f875e02f8baced6e3ec7c5a2782268303e9980a0b8639b2" exitCode=0 Jul 07 15:36:09.215075 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:09.214901 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" event={"ID":"75273620-36d8-4049-8ab9-db420f0fad8a","Type":"ContainerDied","Data":"ac82c5215a1c06e91f875e02f8baced6e3ec7c5a2782268303e9980a0b8639b2"} Jul 07 15:36:09.215360 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:09.215343 2556 scope.go:117] "RemoveContainer" containerID="ac82c5215a1c06e91f875e02f8baced6e3ec7c5a2782268303e9980a0b8639b2" Jul 07 15:36:09.216376 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:09.216347 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:36:09.216448 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:09.216432 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:36:09.216503 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:09.216489 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:36:10.222623 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:10.222587 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" event={"ID":"75273620-36d8-4049-8ab9-db420f0fad8a","Type":"ContainerStarted","Data":"c135d309c6c0c6281c41d32ba8e42fd070910c330774a1f7a9df94496bc2e769"} Jul 07 15:36:10.653879 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:10.653835 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:36:10.654133 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:10.654111 2556 patch_prober.go:28] interesting pod/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 07 15:36:10.654184 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:10.654162 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" podUID="75273620-36d8-4049-8ab9-db420f0fad8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 07 15:36:11.654400 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:11.654367 2556 patch_prober.go:28] interesting pod/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 07 15:36:11.654788 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:11.654425 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" podUID="75273620-36d8-4049-8ab9-db420f0fad8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 07 15:36:12.653757 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:12.653723 2556 patch_prober.go:28] interesting pod/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 07 15:36:12.653948 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:12.653779 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" podUID="75273620-36d8-4049-8ab9-db420f0fad8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 07 15:36:13.654425 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:13.654386 2556 patch_prober.go:28] interesting pod/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 07 15:36:13.654819 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:13.654450 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" podUID="75273620-36d8-4049-8ab9-db420f0fad8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 07 15:36:14.656951 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:14.656923 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:36:14.657329 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:14.657276 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:36:14.658139 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:36:14.658117 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:37:31.010117 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:31.010084 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g"] Jul 07 15:37:31.010657 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:31.010307 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" podUID="f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" containerName="istio-proxy" containerID="cri-o://c6ce9a5a1073f99dcb2ff54c8cbace525bdb58a8d3a523203c0565b1348cd6af" gracePeriod=30 Jul 07 15:37:36.271217 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.271154 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:37:36.431123 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.431094 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-credential-socket\") pod \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " Jul 07 15:37:36.431301 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.431156 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-data\") pod \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " Jul 07 15:37:36.431301 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.431195 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-podinfo\") pod \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " Jul 07 15:37:36.431301 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.431231 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5svqh\" (UniqueName: \"kubernetes.io/projected/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-kube-api-access-5svqh\") pod \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " Jul 07 15:37:36.431301 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.431281 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-token\") pod \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " Jul 07 15:37:36.431487 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.431349 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istiod-ca-cert\") pod \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " Jul 07 15:37:36.431487 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.431383 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-workload-certs\") pod \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " Jul 07 15:37:36.431487 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.431416 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-envoy\") pod \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " Jul 07 15:37:36.431487 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.431442 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" (UID: "f3a2f7b9-250a-4d35-a7b2-13fa56359b6c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:37:36.431487 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.431470 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-workload-socket\") pod \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\" (UID: \"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c\") " Jul 07 15:37:36.431929 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.431719 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" (UID: "f3a2f7b9-250a-4d35-a7b2-13fa56359b6c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:37:36.431929 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.431835 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-credential-socket\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:37:36.431929 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.431854 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-workload-certs\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:37:36.432180 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.431943 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" (UID: "f3a2f7b9-250a-4d35-a7b2-13fa56359b6c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:37:36.432180 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.432043 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" (UID: "f3a2f7b9-250a-4d35-a7b2-13fa56359b6c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:37:36.432296 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.432238 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-data" (OuterVolumeSpecName: "istio-data") pod "f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" (UID: "f3a2f7b9-250a-4d35-a7b2-13fa56359b6c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:37:36.434125 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.434099 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" (UID: "f3a2f7b9-250a-4d35-a7b2-13fa56359b6c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 15:37:36.434125 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.434112 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-kube-api-access-5svqh" (OuterVolumeSpecName: "kube-api-access-5svqh") pod "f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" (UID: "f3a2f7b9-250a-4d35-a7b2-13fa56359b6c"). InnerVolumeSpecName "kube-api-access-5svqh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:37:36.434271 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.434132 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" (UID: "f3a2f7b9-250a-4d35-a7b2-13fa56359b6c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:37:36.434271 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.434139 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-token" (OuterVolumeSpecName: "istio-token") pod "f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" (UID: "f3a2f7b9-250a-4d35-a7b2-13fa56359b6c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:37:36.533185 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.533100 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-data\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:37:36.533185 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.533129 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-podinfo\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:37:36.533185 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.533140 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5svqh\" (UniqueName: \"kubernetes.io/projected/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-kube-api-access-5svqh\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:37:36.533185 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.533150 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-token\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:37:36.533185 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.533159 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istiod-ca-cert\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:37:36.533185 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.533166 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-istio-envoy\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:37:36.533185 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.533173 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c-workload-socket\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:37:36.648755 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.648720 2556 generic.go:356] "Generic (PLEG): container finished" podID="f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" containerID="c6ce9a5a1073f99dcb2ff54c8cbace525bdb58a8d3a523203c0565b1348cd6af" exitCode=0 Jul 07 15:37:36.648975 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.648768 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" Jul 07 15:37:36.648975 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.648802 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" event={"ID":"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c","Type":"ContainerDied","Data":"c6ce9a5a1073f99dcb2ff54c8cbace525bdb58a8d3a523203c0565b1348cd6af"} Jul 07 15:37:36.648975 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.648877 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g" event={"ID":"f3a2f7b9-250a-4d35-a7b2-13fa56359b6c","Type":"ContainerDied","Data":"698133965c625b74a8e05801fafb07b0f6c9dce70f962493ad156a999242f15e"} Jul 07 15:37:36.648975 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.648900 2556 scope.go:117] "RemoveContainer" containerID="c6ce9a5a1073f99dcb2ff54c8cbace525bdb58a8d3a523203c0565b1348cd6af" Jul 07 15:37:36.660518 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.660479 2556 scope.go:117] "RemoveContainer" containerID="c538e8985611e73702ada440c58593ff8931283fd5599ef5070707389cc06646" Jul 07 15:37:36.669448 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.669424 2556 scope.go:117] "RemoveContainer" containerID="c6ce9a5a1073f99dcb2ff54c8cbace525bdb58a8d3a523203c0565b1348cd6af" Jul 07 15:37:36.669764 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:37:36.669736 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6ce9a5a1073f99dcb2ff54c8cbace525bdb58a8d3a523203c0565b1348cd6af\": container with ID starting with c6ce9a5a1073f99dcb2ff54c8cbace525bdb58a8d3a523203c0565b1348cd6af not found: ID does not exist" containerID="c6ce9a5a1073f99dcb2ff54c8cbace525bdb58a8d3a523203c0565b1348cd6af" Jul 07 15:37:36.669833 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.669777 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6ce9a5a1073f99dcb2ff54c8cbace525bdb58a8d3a523203c0565b1348cd6af"} err="failed to get container status \"c6ce9a5a1073f99dcb2ff54c8cbace525bdb58a8d3a523203c0565b1348cd6af\": rpc error: code = NotFound desc = could not find container \"c6ce9a5a1073f99dcb2ff54c8cbace525bdb58a8d3a523203c0565b1348cd6af\": container with ID starting with c6ce9a5a1073f99dcb2ff54c8cbace525bdb58a8d3a523203c0565b1348cd6af not found: ID does not exist" Jul 07 15:37:36.669833 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.669812 2556 scope.go:117] "RemoveContainer" containerID="c538e8985611e73702ada440c58593ff8931283fd5599ef5070707389cc06646" Jul 07 15:37:36.670144 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:37:36.670124 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c538e8985611e73702ada440c58593ff8931283fd5599ef5070707389cc06646\": container with ID starting with c538e8985611e73702ada440c58593ff8931283fd5599ef5070707389cc06646 not found: ID does not exist" containerID="c538e8985611e73702ada440c58593ff8931283fd5599ef5070707389cc06646" Jul 07 15:37:36.670211 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.670151 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c538e8985611e73702ada440c58593ff8931283fd5599ef5070707389cc06646"} err="failed to get container status \"c538e8985611e73702ada440c58593ff8931283fd5599ef5070707389cc06646\": rpc error: code = NotFound desc = could not find container \"c538e8985611e73702ada440c58593ff8931283fd5599ef5070707389cc06646\": container with ID starting with c538e8985611e73702ada440c58593ff8931283fd5599ef5070707389cc06646 not found: ID does not exist" Jul 07 15:37:36.705721 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.705686 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g"] Jul 07 15:37:36.724923 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:36.724886 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-6c4fac-openshift-default-7cfc8c44c9-94x2g"] Jul 07 15:37:37.788690 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:37.788644 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" path="/var/lib/kubelet/pods/f3a2f7b9-250a-4d35-a7b2-13fa56359b6c/volumes" Jul 07 15:37:38.133309 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:38.133279 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb"] Jul 07 15:37:38.133485 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:38.133468 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" podUID="75273620-36d8-4049-8ab9-db420f0fad8a" containerName="istio-proxy" containerID="cri-o://c135d309c6c0c6281c41d32ba8e42fd070910c330774a1f7a9df94496bc2e769" gracePeriod=30 Jul 07 15:37:39.112682 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.112647 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx"] Jul 07 15:37:39.113061 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.113018 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" containerName="istio-proxy" Jul 07 15:37:39.113061 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.113031 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" containerName="istio-proxy" Jul 07 15:37:39.113061 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.113041 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" containerName="istio-proxy" Jul 07 15:37:39.113061 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.113047 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" containerName="istio-proxy" Jul 07 15:37:39.113198 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.113107 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" containerName="istio-proxy" Jul 07 15:37:39.113198 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.113116 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3a2f7b9-250a-4d35-a7b2-13fa56359b6c" containerName="istio-proxy" Jul 07 15:37:39.117629 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.117596 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.120042 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.120025 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ad5677-openshift-default-dockercfg-hpmpn\"" Jul 07 15:37:39.130608 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.130584 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx"] Jul 07 15:37:39.261457 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.261426 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-envoy\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.261637 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.261461 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istiod-ca-cert\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.261637 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.261490 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-workload-socket\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.261637 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.261514 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkdr8\" (UniqueName: \"kubernetes.io/projected/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-kube-api-access-gkdr8\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.261637 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.261540 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-podinfo\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.261637 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.261558 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-data\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.261829 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.261652 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-token\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.261829 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.261689 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-credential-socket\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.261829 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.261761 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-workload-certs\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.363017 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.362932 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-workload-certs\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.363017 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.362990 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-envoy\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.363017 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.363015 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istiod-ca-cert\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.363380 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.363032 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-workload-socket\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.363380 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.363059 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkdr8\" (UniqueName: \"kubernetes.io/projected/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-kube-api-access-gkdr8\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.363380 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.363092 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-podinfo\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.363380 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.363110 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-data\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.363380 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.363173 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-token\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.363380 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.363202 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-credential-socket\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.363700 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.363415 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-workload-certs\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.363700 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.363452 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-workload-socket\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.363700 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.363581 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-credential-socket\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.363860 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.363711 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-data\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.363860 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.363753 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istiod-ca-cert\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.365338 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.365317 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-envoy\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.365571 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.365552 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-podinfo\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.372107 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.372085 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-token\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.372300 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.372283 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkdr8\" (UniqueName: \"kubernetes.io/projected/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-kube-api-access-gkdr8\") pod \"e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.428392 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.428357 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:39.567580 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.567552 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx"] Jul 07 15:37:39.569485 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:37:39.569459 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b1c3617_18f0_4451_96da_94f3a3fc2cc8.slice/crio-c08f6473e9147299d78d7892b1571e734070ae8d284625713141d13db2eaff73 WatchSource:0}: Error finding container c08f6473e9147299d78d7892b1571e734070ae8d284625713141d13db2eaff73: Status 404 returned error can't find the container with id c08f6473e9147299d78d7892b1571e734070ae8d284625713141d13db2eaff73 Jul 07 15:37:39.572010 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.571970 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:37:39.572119 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.572061 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:37:39.572119 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.572106 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:37:39.667191 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.667151 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" event={"ID":"8b1c3617-18f0-4451-96da-94f3a3fc2cc8","Type":"ContainerStarted","Data":"f971ef862c8ad4370170ac1c784c44cc9ec7094f247b215db49dfbff4f0d8d6b"} Jul 07 15:37:39.667383 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.667219 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" event={"ID":"8b1c3617-18f0-4451-96da-94f3a3fc2cc8","Type":"ContainerStarted","Data":"c08f6473e9147299d78d7892b1571e734070ae8d284625713141d13db2eaff73"} Jul 07 15:37:39.695080 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:39.694995 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" podStartSLOduration=0.69495565 podStartE2EDuration="694.95565ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:37:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:37:39.692443491 +0000 UTC m=+2208.508709523" watchObservedRunningTime="2026-07-07 15:37:39.69495565 +0000 UTC m=+2208.511221694" Jul 07 15:37:40.429580 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:40.429544 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:40.434980 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:40.434955 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:40.672356 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:40.672319 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:40.673213 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:40.673195 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:37:43.385277 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.385251 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:37:43.500107 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.500019 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75273620-36d8-4049-8ab9-db420f0fad8a-istio-podinfo\") pod \"75273620-36d8-4049-8ab9-db420f0fad8a\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " Jul 07 15:37:43.500107 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.500076 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75273620-36d8-4049-8ab9-db420f0fad8a-istiod-ca-cert\") pod \"75273620-36d8-4049-8ab9-db420f0fad8a\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " Jul 07 15:37:43.500107 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.500111 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-istio-data\") pod \"75273620-36d8-4049-8ab9-db420f0fad8a\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " Jul 07 15:37:43.500403 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.500134 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkc66\" (UniqueName: \"kubernetes.io/projected/75273620-36d8-4049-8ab9-db420f0fad8a-kube-api-access-hkc66\") pod \"75273620-36d8-4049-8ab9-db420f0fad8a\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " Jul 07 15:37:43.500403 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.500186 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75273620-36d8-4049-8ab9-db420f0fad8a-istio-token\") pod \"75273620-36d8-4049-8ab9-db420f0fad8a\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " Jul 07 15:37:43.500403 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.500218 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-credential-socket\") pod \"75273620-36d8-4049-8ab9-db420f0fad8a\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " Jul 07 15:37:43.500403 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.500277 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-workload-certs\") pod \"75273620-36d8-4049-8ab9-db420f0fad8a\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " Jul 07 15:37:43.500403 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.500316 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-workload-socket\") pod \"75273620-36d8-4049-8ab9-db420f0fad8a\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " Jul 07 15:37:43.500403 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.500381 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-istio-envoy\") pod \"75273620-36d8-4049-8ab9-db420f0fad8a\" (UID: \"75273620-36d8-4049-8ab9-db420f0fad8a\") " Jul 07 15:37:43.500681 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.500582 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75273620-36d8-4049-8ab9-db420f0fad8a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "75273620-36d8-4049-8ab9-db420f0fad8a" (UID: "75273620-36d8-4049-8ab9-db420f0fad8a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:37:43.500681 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.500646 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "75273620-36d8-4049-8ab9-db420f0fad8a" (UID: "75273620-36d8-4049-8ab9-db420f0fad8a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:37:43.500782 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.500744 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-credential-socket\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:37:43.500782 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.500765 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75273620-36d8-4049-8ab9-db420f0fad8a-istiod-ca-cert\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:37:43.500919 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.500794 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "75273620-36d8-4049-8ab9-db420f0fad8a" (UID: "75273620-36d8-4049-8ab9-db420f0fad8a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:37:43.500919 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.500809 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "75273620-36d8-4049-8ab9-db420f0fad8a" (UID: "75273620-36d8-4049-8ab9-db420f0fad8a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:37:43.501140 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.501112 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-istio-data" (OuterVolumeSpecName: "istio-data") pod "75273620-36d8-4049-8ab9-db420f0fad8a" (UID: "75273620-36d8-4049-8ab9-db420f0fad8a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:37:43.502469 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.502441 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/75273620-36d8-4049-8ab9-db420f0fad8a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "75273620-36d8-4049-8ab9-db420f0fad8a" (UID: "75273620-36d8-4049-8ab9-db420f0fad8a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 15:37:43.502589 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.502480 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75273620-36d8-4049-8ab9-db420f0fad8a-kube-api-access-hkc66" (OuterVolumeSpecName: "kube-api-access-hkc66") pod "75273620-36d8-4049-8ab9-db420f0fad8a" (UID: "75273620-36d8-4049-8ab9-db420f0fad8a"). InnerVolumeSpecName "kube-api-access-hkc66". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:37:43.502804 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.502783 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "75273620-36d8-4049-8ab9-db420f0fad8a" (UID: "75273620-36d8-4049-8ab9-db420f0fad8a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:37:43.502898 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.502838 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75273620-36d8-4049-8ab9-db420f0fad8a-istio-token" (OuterVolumeSpecName: "istio-token") pod "75273620-36d8-4049-8ab9-db420f0fad8a" (UID: "75273620-36d8-4049-8ab9-db420f0fad8a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:37:43.601288 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.601249 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-workload-certs\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:37:43.601288 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.601280 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-workload-socket\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:37:43.601288 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.601289 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-istio-envoy\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:37:43.601288 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.601297 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75273620-36d8-4049-8ab9-db420f0fad8a-istio-podinfo\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:37:43.601641 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.601305 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75273620-36d8-4049-8ab9-db420f0fad8a-istio-data\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:37:43.601641 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.601313 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkc66\" (UniqueName: \"kubernetes.io/projected/75273620-36d8-4049-8ab9-db420f0fad8a-kube-api-access-hkc66\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:37:43.601641 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.601333 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75273620-36d8-4049-8ab9-db420f0fad8a-istio-token\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:37:43.688652 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.688613 2556 generic.go:356] "Generic (PLEG): container finished" podID="75273620-36d8-4049-8ab9-db420f0fad8a" containerID="c135d309c6c0c6281c41d32ba8e42fd070910c330774a1f7a9df94496bc2e769" exitCode=0 Jul 07 15:37:43.688817 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.688662 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" Jul 07 15:37:43.688817 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.688660 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" event={"ID":"75273620-36d8-4049-8ab9-db420f0fad8a","Type":"ContainerDied","Data":"c135d309c6c0c6281c41d32ba8e42fd070910c330774a1f7a9df94496bc2e769"} Jul 07 15:37:43.688817 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.688787 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb" event={"ID":"75273620-36d8-4049-8ab9-db420f0fad8a","Type":"ContainerDied","Data":"12af435b387c949856e4cb546e6e1b0ce6791af72602a4662012fe3423214131"} Jul 07 15:37:43.688964 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.688820 2556 scope.go:117] "RemoveContainer" containerID="c135d309c6c0c6281c41d32ba8e42fd070910c330774a1f7a9df94496bc2e769" Jul 07 15:37:43.699128 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.699086 2556 scope.go:117] "RemoveContainer" containerID="ac82c5215a1c06e91f875e02f8baced6e3ec7c5a2782268303e9980a0b8639b2" Jul 07 15:37:43.707357 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.707340 2556 scope.go:117] "RemoveContainer" containerID="c135d309c6c0c6281c41d32ba8e42fd070910c330774a1f7a9df94496bc2e769" Jul 07 15:37:43.707620 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:37:43.707600 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c135d309c6c0c6281c41d32ba8e42fd070910c330774a1f7a9df94496bc2e769\": container with ID starting with c135d309c6c0c6281c41d32ba8e42fd070910c330774a1f7a9df94496bc2e769 not found: ID does not exist" containerID="c135d309c6c0c6281c41d32ba8e42fd070910c330774a1f7a9df94496bc2e769" Jul 07 15:37:43.707668 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.707628 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c135d309c6c0c6281c41d32ba8e42fd070910c330774a1f7a9df94496bc2e769"} err="failed to get container status \"c135d309c6c0c6281c41d32ba8e42fd070910c330774a1f7a9df94496bc2e769\": rpc error: code = NotFound desc = could not find container \"c135d309c6c0c6281c41d32ba8e42fd070910c330774a1f7a9df94496bc2e769\": container with ID starting with c135d309c6c0c6281c41d32ba8e42fd070910c330774a1f7a9df94496bc2e769 not found: ID does not exist" Jul 07 15:37:43.707668 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.707646 2556 scope.go:117] "RemoveContainer" containerID="ac82c5215a1c06e91f875e02f8baced6e3ec7c5a2782268303e9980a0b8639b2" Jul 07 15:37:43.707907 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:37:43.707887 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac82c5215a1c06e91f875e02f8baced6e3ec7c5a2782268303e9980a0b8639b2\": container with ID starting with ac82c5215a1c06e91f875e02f8baced6e3ec7c5a2782268303e9980a0b8639b2 not found: ID does not exist" containerID="ac82c5215a1c06e91f875e02f8baced6e3ec7c5a2782268303e9980a0b8639b2" Jul 07 15:37:43.707975 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.707916 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac82c5215a1c06e91f875e02f8baced6e3ec7c5a2782268303e9980a0b8639b2"} err="failed to get container status \"ac82c5215a1c06e91f875e02f8baced6e3ec7c5a2782268303e9980a0b8639b2\": rpc error: code = NotFound desc = could not find container \"ac82c5215a1c06e91f875e02f8baced6e3ec7c5a2782268303e9980a0b8639b2\": container with ID starting with ac82c5215a1c06e91f875e02f8baced6e3ec7c5a2782268303e9980a0b8639b2 not found: ID does not exist" Jul 07 15:37:43.713538 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.713513 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb"] Jul 07 15:37:43.719209 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.719187 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-c5eea1-openshift-default-674cf5b95f-6nbrb"] Jul 07 15:37:43.788451 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:43.788375 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75273620-36d8-4049-8ab9-db420f0fad8a" path="/var/lib/kubelet/pods/75273620-36d8-4049-8ab9-db420f0fad8a/volumes" Jul 07 15:37:47.049445 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.049405 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds"] Jul 07 15:37:47.049940 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.049785 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75273620-36d8-4049-8ab9-db420f0fad8a" containerName="istio-proxy" Jul 07 15:37:47.049940 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.049799 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="75273620-36d8-4049-8ab9-db420f0fad8a" containerName="istio-proxy" Jul 07 15:37:47.049940 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.049805 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75273620-36d8-4049-8ab9-db420f0fad8a" containerName="istio-proxy" Jul 07 15:37:47.049940 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.049811 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="75273620-36d8-4049-8ab9-db420f0fad8a" containerName="istio-proxy" Jul 07 15:37:47.049940 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.049899 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="75273620-36d8-4049-8ab9-db420f0fad8a" containerName="istio-proxy" Jul 07 15:37:47.054444 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.054421 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.057130 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.057110 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-83bd55-openshift-default-dockercfg-7c9nd\"" Jul 07 15:37:47.073469 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.073443 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds"] Jul 07 15:37:47.238318 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.238280 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-envoy\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.238501 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.238323 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-data\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.238501 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.238360 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3209d0e-fc74-414e-8ffa-190078566e1e-istiod-ca-cert\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.238501 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.238402 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-workload-certs\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.238501 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.238429 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-workload-socket\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.238501 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.238472 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-token\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.238679 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.238536 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-podinfo\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.238679 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.238570 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp2sq\" (UniqueName: \"kubernetes.io/projected/c3209d0e-fc74-414e-8ffa-190078566e1e-kube-api-access-hp2sq\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.238679 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.238672 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-credential-socket\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.340136 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.340102 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-envoy\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.340136 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.340138 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-data\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.340403 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.340159 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3209d0e-fc74-414e-8ffa-190078566e1e-istiod-ca-cert\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.340403 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.340195 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-workload-certs\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.340403 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.340224 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-workload-socket\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.340403 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.340252 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-token\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.340403 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.340280 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-podinfo\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.340403 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.340308 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp2sq\" (UniqueName: \"kubernetes.io/projected/c3209d0e-fc74-414e-8ffa-190078566e1e-kube-api-access-hp2sq\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.340706 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.340419 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-credential-socket\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.340706 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.340657 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-workload-certs\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.340810 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.340757 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-data\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.340860 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.340803 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-workload-socket\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.340923 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.340859 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-credential-socket\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.341060 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.341041 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3209d0e-fc74-414e-8ffa-190078566e1e-istiod-ca-cert\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.342472 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.342448 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-envoy\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.342960 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.342942 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-podinfo\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.348660 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.348635 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-token\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.348837 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.348811 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp2sq\" (UniqueName: \"kubernetes.io/projected/c3209d0e-fc74-414e-8ffa-190078566e1e-kube-api-access-hp2sq\") pod \"e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.364717 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.364694 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:47.520189 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.520163 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds"] Jul 07 15:37:47.522327 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:37:47.522294 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3209d0e_fc74_414e_8ffa_190078566e1e.slice/crio-dce3b0ac6449aaff71ca21566318b6968658b7d3056903defc00c7a14aefc63c WatchSource:0}: Error finding container dce3b0ac6449aaff71ca21566318b6968658b7d3056903defc00c7a14aefc63c: Status 404 returned error can't find the container with id dce3b0ac6449aaff71ca21566318b6968658b7d3056903defc00c7a14aefc63c Jul 07 15:37:47.524808 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.524767 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:37:47.524905 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.524853 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:37:47.524953 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.524923 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:37:47.710931 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.710843 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" event={"ID":"c3209d0e-fc74-414e-8ffa-190078566e1e","Type":"ContainerStarted","Data":"4ddf2988c9a7099693ec686d4422195fd55c6566cf420140baf6628dfaead1f7"} Jul 07 15:37:47.710931 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.710919 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" event={"ID":"c3209d0e-fc74-414e-8ffa-190078566e1e","Type":"ContainerStarted","Data":"dce3b0ac6449aaff71ca21566318b6968658b7d3056903defc00c7a14aefc63c"} Jul 07 15:37:47.744324 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:47.744271 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" podStartSLOduration=0.744255265 podStartE2EDuration="744.255265ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:37:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:37:47.742830011 +0000 UTC m=+2216.559096041" watchObservedRunningTime="2026-07-07 15:37:47.744255265 +0000 UTC m=+2216.560521362" Jul 07 15:37:48.365725 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:48.365677 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:48.370976 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:48.370949 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:48.716344 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:48.716266 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:48.717228 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:48.717210 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:37:56.714062 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:56.714016 2556 patch_prober.go:28] interesting pod/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 15:37:56.714474 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:56.714093 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" podUID="8b1c3617-18f0-4451-96da-94f3a3fc2cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 15:37:58.771300 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:58.771266 2556 generic.go:356] "Generic (PLEG): container finished" podID="8b1c3617-18f0-4451-96da-94f3a3fc2cc8" containerID="f971ef862c8ad4370170ac1c784c44cc9ec7094f247b215db49dfbff4f0d8d6b" exitCode=0 Jul 07 15:37:58.771723 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:58.771341 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" event={"ID":"8b1c3617-18f0-4451-96da-94f3a3fc2cc8","Type":"ContainerDied","Data":"f971ef862c8ad4370170ac1c784c44cc9ec7094f247b215db49dfbff4f0d8d6b"} Jul 07 15:37:58.771767 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:58.771747 2556 scope.go:117] "RemoveContainer" containerID="f971ef862c8ad4370170ac1c784c44cc9ec7094f247b215db49dfbff4f0d8d6b" Jul 07 15:37:58.772710 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:58.772680 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:37:58.772807 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:58.772760 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:37:58.772892 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:58.772818 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:37:59.781049 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:37:59.781014 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" event={"ID":"8b1c3617-18f0-4451-96da-94f3a3fc2cc8","Type":"ContainerStarted","Data":"b5b53dcc7639f4a3d6da3e142e5f8bb7e6d1344fabe8def26d9b9de178d25bfc"} Jul 07 15:38:00.429642 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:00.429598 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:38:00.429642 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:00.429638 2556 patch_prober.go:28] interesting pod/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 07 15:38:00.429887 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:00.429680 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" podUID="8b1c3617-18f0-4451-96da-94f3a3fc2cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 07 15:38:01.429705 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:01.429665 2556 patch_prober.go:28] interesting pod/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 07 15:38:01.430157 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:01.429727 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" podUID="8b1c3617-18f0-4451-96da-94f3a3fc2cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 07 15:38:02.429297 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:02.429256 2556 patch_prober.go:28] interesting pod/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 07 15:38:02.429553 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:02.429311 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" podUID="8b1c3617-18f0-4451-96da-94f3a3fc2cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 07 15:38:03.429709 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:03.429673 2556 patch_prober.go:28] interesting pod/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 07 15:38:03.430095 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:03.429745 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" podUID="8b1c3617-18f0-4451-96da-94f3a3fc2cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 07 15:38:04.433785 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:04.433754 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:38:04.434244 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:04.434077 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:38:04.434987 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:04.434966 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:38:04.758130 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:04.758030 2556 patch_prober.go:28] interesting pod/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 15:38:04.758130 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:04.758106 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" podUID="c3209d0e-fc74-414e-8ffa-190078566e1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 15:38:06.827960 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:06.827929 2556 generic.go:356] "Generic (PLEG): container finished" podID="c3209d0e-fc74-414e-8ffa-190078566e1e" containerID="4ddf2988c9a7099693ec686d4422195fd55c6566cf420140baf6628dfaead1f7" exitCode=0 Jul 07 15:38:06.828565 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:06.828013 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" event={"ID":"c3209d0e-fc74-414e-8ffa-190078566e1e","Type":"ContainerDied","Data":"4ddf2988c9a7099693ec686d4422195fd55c6566cf420140baf6628dfaead1f7"} Jul 07 15:38:06.828565 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:06.828460 2556 scope.go:117] "RemoveContainer" containerID="4ddf2988c9a7099693ec686d4422195fd55c6566cf420140baf6628dfaead1f7" Jul 07 15:38:06.829513 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:06.829481 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:38:06.829621 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:06.829553 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:38:06.829621 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:06.829591 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:38:07.840118 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:07.840086 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" event={"ID":"c3209d0e-fc74-414e-8ffa-190078566e1e","Type":"ContainerStarted","Data":"7e4e56479883537de436dcd6eed7c1d8002c69a1d083828b33bb1fbbf52b573a"} Jul 07 15:38:08.365307 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:08.365270 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:38:08.365568 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:08.365541 2556 patch_prober.go:28] interesting pod/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 07 15:38:08.365664 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:08.365598 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" podUID="c3209d0e-fc74-414e-8ffa-190078566e1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 07 15:38:09.292645 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:09.292612 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx"] Jul 07 15:38:09.293428 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:09.293397 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" podUID="8b1c3617-18f0-4451-96da-94f3a3fc2cc8" containerName="istio-proxy" containerID="cri-o://b5b53dcc7639f4a3d6da3e142e5f8bb7e6d1344fabe8def26d9b9de178d25bfc" gracePeriod=30 Jul 07 15:38:09.372894 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:09.365534 2556 patch_prober.go:28] interesting pod/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 07 15:38:09.372894 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:09.365609 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" podUID="c3209d0e-fc74-414e-8ffa-190078566e1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 07 15:38:10.366132 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:10.365624 2556 patch_prober.go:28] interesting pod/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 07 15:38:10.366132 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:10.365693 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" podUID="c3209d0e-fc74-414e-8ffa-190078566e1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 07 15:38:11.366536 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:11.366047 2556 patch_prober.go:28] interesting pod/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 07 15:38:11.366536 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:11.366116 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" podUID="c3209d0e-fc74-414e-8ffa-190078566e1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 07 15:38:12.368670 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:12.368638 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:38:12.369113 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:12.368920 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:38:12.369705 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:12.369688 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:38:14.555641 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.555581 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:38:14.643889 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.643829 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-podinfo\") pod \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " Jul 07 15:38:14.644078 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.643935 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-workload-socket\") pod \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " Jul 07 15:38:14.644078 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.643974 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-data\") pod \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " Jul 07 15:38:14.644078 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.644027 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkdr8\" (UniqueName: \"kubernetes.io/projected/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-kube-api-access-gkdr8\") pod \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " Jul 07 15:38:14.644078 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.644071 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istiod-ca-cert\") pod \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " Jul 07 15:38:14.644295 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.644112 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-envoy\") pod \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " Jul 07 15:38:14.644295 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.644138 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-token\") pod \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " Jul 07 15:38:14.644295 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.644198 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-workload-certs\") pod \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " Jul 07 15:38:14.644295 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.644234 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-credential-socket\") pod \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\" (UID: \"8b1c3617-18f0-4451-96da-94f3a3fc2cc8\") " Jul 07 15:38:14.644499 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.644298 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8b1c3617-18f0-4451-96da-94f3a3fc2cc8" (UID: "8b1c3617-18f0-4451-96da-94f3a3fc2cc8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:38:14.644556 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.644512 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8b1c3617-18f0-4451-96da-94f3a3fc2cc8" (UID: "8b1c3617-18f0-4451-96da-94f3a3fc2cc8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:38:14.644556 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.644533 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8b1c3617-18f0-4451-96da-94f3a3fc2cc8" (UID: "8b1c3617-18f0-4451-96da-94f3a3fc2cc8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:38:14.644654 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.644565 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-workload-socket\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:38:14.644769 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.644744 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8b1c3617-18f0-4451-96da-94f3a3fc2cc8" (UID: "8b1c3617-18f0-4451-96da-94f3a3fc2cc8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:38:14.644938 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.644916 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-data" (OuterVolumeSpecName: "istio-data") pod "8b1c3617-18f0-4451-96da-94f3a3fc2cc8" (UID: "8b1c3617-18f0-4451-96da-94f3a3fc2cc8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:38:14.646374 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.646343 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-kube-api-access-gkdr8" (OuterVolumeSpecName: "kube-api-access-gkdr8") pod "8b1c3617-18f0-4451-96da-94f3a3fc2cc8" (UID: "8b1c3617-18f0-4451-96da-94f3a3fc2cc8"). InnerVolumeSpecName "kube-api-access-gkdr8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:38:14.646374 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.646359 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8b1c3617-18f0-4451-96da-94f3a3fc2cc8" (UID: "8b1c3617-18f0-4451-96da-94f3a3fc2cc8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 15:38:14.646506 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.646393 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8b1c3617-18f0-4451-96da-94f3a3fc2cc8" (UID: "8b1c3617-18f0-4451-96da-94f3a3fc2cc8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:38:14.646506 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.646472 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-token" (OuterVolumeSpecName: "istio-token") pod "8b1c3617-18f0-4451-96da-94f3a3fc2cc8" (UID: "8b1c3617-18f0-4451-96da-94f3a3fc2cc8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:38:14.745135 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.745097 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istiod-ca-cert\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:38:14.745135 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.745127 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-envoy\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:38:14.745135 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.745136 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-token\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:38:14.745135 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.745144 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-workload-certs\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:38:14.745390 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.745151 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-credential-socket\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:38:14.745390 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.745160 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-podinfo\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:38:14.745390 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.745168 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-istio-data\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:38:14.745390 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.745175 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkdr8\" (UniqueName: \"kubernetes.io/projected/8b1c3617-18f0-4451-96da-94f3a3fc2cc8-kube-api-access-gkdr8\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:38:14.874849 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.874815 2556 generic.go:356] "Generic (PLEG): container finished" podID="8b1c3617-18f0-4451-96da-94f3a3fc2cc8" containerID="b5b53dcc7639f4a3d6da3e142e5f8bb7e6d1344fabe8def26d9b9de178d25bfc" exitCode=0 Jul 07 15:38:14.875051 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.874863 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" Jul 07 15:38:14.875051 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.874911 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" event={"ID":"8b1c3617-18f0-4451-96da-94f3a3fc2cc8","Type":"ContainerDied","Data":"b5b53dcc7639f4a3d6da3e142e5f8bb7e6d1344fabe8def26d9b9de178d25bfc"} Jul 07 15:38:14.875051 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.874974 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx" event={"ID":"8b1c3617-18f0-4451-96da-94f3a3fc2cc8","Type":"ContainerDied","Data":"c08f6473e9147299d78d7892b1571e734070ae8d284625713141d13db2eaff73"} Jul 07 15:38:14.875051 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.874999 2556 scope.go:117] "RemoveContainer" containerID="b5b53dcc7639f4a3d6da3e142e5f8bb7e6d1344fabe8def26d9b9de178d25bfc" Jul 07 15:38:14.884639 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.884618 2556 scope.go:117] "RemoveContainer" containerID="f971ef862c8ad4370170ac1c784c44cc9ec7094f247b215db49dfbff4f0d8d6b" Jul 07 15:38:14.892821 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.892803 2556 scope.go:117] "RemoveContainer" containerID="b5b53dcc7639f4a3d6da3e142e5f8bb7e6d1344fabe8def26d9b9de178d25bfc" Jul 07 15:38:14.893083 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:38:14.893062 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5b53dcc7639f4a3d6da3e142e5f8bb7e6d1344fabe8def26d9b9de178d25bfc\": container with ID starting with b5b53dcc7639f4a3d6da3e142e5f8bb7e6d1344fabe8def26d9b9de178d25bfc not found: ID does not exist" containerID="b5b53dcc7639f4a3d6da3e142e5f8bb7e6d1344fabe8def26d9b9de178d25bfc" Jul 07 15:38:14.893155 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.893093 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5b53dcc7639f4a3d6da3e142e5f8bb7e6d1344fabe8def26d9b9de178d25bfc"} err="failed to get container status \"b5b53dcc7639f4a3d6da3e142e5f8bb7e6d1344fabe8def26d9b9de178d25bfc\": rpc error: code = NotFound desc = could not find container \"b5b53dcc7639f4a3d6da3e142e5f8bb7e6d1344fabe8def26d9b9de178d25bfc\": container with ID starting with b5b53dcc7639f4a3d6da3e142e5f8bb7e6d1344fabe8def26d9b9de178d25bfc not found: ID does not exist" Jul 07 15:38:14.893155 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.893111 2556 scope.go:117] "RemoveContainer" containerID="f971ef862c8ad4370170ac1c784c44cc9ec7094f247b215db49dfbff4f0d8d6b" Jul 07 15:38:14.893349 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:38:14.893327 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f971ef862c8ad4370170ac1c784c44cc9ec7094f247b215db49dfbff4f0d8d6b\": container with ID starting with f971ef862c8ad4370170ac1c784c44cc9ec7094f247b215db49dfbff4f0d8d6b not found: ID does not exist" containerID="f971ef862c8ad4370170ac1c784c44cc9ec7094f247b215db49dfbff4f0d8d6b" Jul 07 15:38:14.893385 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.893356 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f971ef862c8ad4370170ac1c784c44cc9ec7094f247b215db49dfbff4f0d8d6b"} err="failed to get container status \"f971ef862c8ad4370170ac1c784c44cc9ec7094f247b215db49dfbff4f0d8d6b\": rpc error: code = NotFound desc = could not find container \"f971ef862c8ad4370170ac1c784c44cc9ec7094f247b215db49dfbff4f0d8d6b\": container with ID starting with f971ef862c8ad4370170ac1c784c44cc9ec7094f247b215db49dfbff4f0d8d6b not found: ID does not exist" Jul 07 15:38:14.899099 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.899075 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx"] Jul 07 15:38:14.905887 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:14.905846 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ad5677-openshift-default-58f5fdcf87-6pmhx"] Jul 07 15:38:15.788265 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:15.788234 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b1c3617-18f0-4451-96da-94f3a3fc2cc8" path="/var/lib/kubelet/pods/8b1c3617-18f0-4451-96da-94f3a3fc2cc8/volumes" Jul 07 15:38:16.453343 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:16.453304 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds"] Jul 07 15:38:16.453599 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:16.453571 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" podUID="c3209d0e-fc74-414e-8ffa-190078566e1e" containerName="istio-proxy" containerID="cri-o://7e4e56479883537de436dcd6eed7c1d8002c69a1d083828b33bb1fbbf52b573a" gracePeriod=30 Jul 07 15:38:17.434327 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.434290 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v"] Jul 07 15:38:17.434849 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.434828 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b1c3617-18f0-4451-96da-94f3a3fc2cc8" containerName="istio-proxy" Jul 07 15:38:17.434961 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.434852 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b1c3617-18f0-4451-96da-94f3a3fc2cc8" containerName="istio-proxy" Jul 07 15:38:17.434961 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.434915 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b1c3617-18f0-4451-96da-94f3a3fc2cc8" containerName="istio-proxy" Jul 07 15:38:17.434961 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.434926 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b1c3617-18f0-4451-96da-94f3a3fc2cc8" containerName="istio-proxy" Jul 07 15:38:17.435117 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.435027 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b1c3617-18f0-4451-96da-94f3a3fc2cc8" containerName="istio-proxy" Jul 07 15:38:17.435117 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.435043 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b1c3617-18f0-4451-96da-94f3a3fc2cc8" containerName="istio-proxy" Jul 07 15:38:17.435117 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.435054 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="75273620-36d8-4049-8ab9-db420f0fad8a" containerName="istio-proxy" Jul 07 15:38:17.438630 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.438608 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.441037 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.441014 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-f09258-openshift-default-dockercfg-k75jz\"" Jul 07 15:38:17.449443 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.449419 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v"] Jul 07 15:38:17.574101 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.574043 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-data\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.574298 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.574133 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-podinfo\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.574298 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.574167 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11d693e8-dd5f-4fd0-8192-5410a3539741-istiod-ca-cert\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.574298 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.574197 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-credential-socket\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.574298 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.574272 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-workload-certs\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.574442 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.574342 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-envoy\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.574442 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.574391 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-workload-socket\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.574442 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.574417 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-token\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.574442 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.574437 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82bh9\" (UniqueName: \"kubernetes.io/projected/11d693e8-dd5f-4fd0-8192-5410a3539741-kube-api-access-82bh9\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.675757 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.675720 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-workload-certs\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.675757 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.675769 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-envoy\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.676065 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.675793 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-workload-socket\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.676065 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.675820 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-token\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.676065 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.675849 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82bh9\" (UniqueName: \"kubernetes.io/projected/11d693e8-dd5f-4fd0-8192-5410a3539741-kube-api-access-82bh9\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.676065 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.675919 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-data\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.676284 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.676146 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-podinfo\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.676284 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.676195 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11d693e8-dd5f-4fd0-8192-5410a3539741-istiod-ca-cert\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.676284 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.676212 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-workload-certs\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.676284 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.676227 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-credential-socket\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.676489 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.676287 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-workload-socket\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.676489 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.676365 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-data\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.676594 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.676537 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-credential-socket\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.676971 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.676941 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11d693e8-dd5f-4fd0-8192-5410a3539741-istiod-ca-cert\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.678620 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.678593 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-envoy\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.678723 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.678626 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-podinfo\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.684362 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.684295 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82bh9\" (UniqueName: \"kubernetes.io/projected/11d693e8-dd5f-4fd0-8192-5410a3539741-kube-api-access-82bh9\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.684442 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.684431 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-token\") pod \"e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.750491 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.750454 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:17.890412 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.890387 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v"] Jul 07 15:38:17.892755 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:38:17.892726 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d693e8_dd5f_4fd0_8192_5410a3539741.slice/crio-1f859acae38beb1b601d505b22190a027e5a0471bf181cba04a3d470daac0b8e WatchSource:0}: Error finding container 1f859acae38beb1b601d505b22190a027e5a0471bf181cba04a3d470daac0b8e: Status 404 returned error can't find the container with id 1f859acae38beb1b601d505b22190a027e5a0471bf181cba04a3d470daac0b8e Jul 07 15:38:17.894884 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.894837 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:38:17.894980 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.894941 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:38:17.895051 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:17.894990 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:38:18.899402 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:18.899370 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" event={"ID":"11d693e8-dd5f-4fd0-8192-5410a3539741","Type":"ContainerStarted","Data":"0d8e97ad091050ca70cf72daaa86998c05f1e5b4670f19780eb1348d85c0dc10"} Jul 07 15:38:18.899785 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:18.899434 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" event={"ID":"11d693e8-dd5f-4fd0-8192-5410a3539741","Type":"ContainerStarted","Data":"1f859acae38beb1b601d505b22190a027e5a0471bf181cba04a3d470daac0b8e"} Jul 07 15:38:18.928320 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:18.928264 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" podStartSLOduration=1.928246761 podStartE2EDuration="1.928246761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:38:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:38:18.925033637 +0000 UTC m=+2247.741299690" watchObservedRunningTime="2026-07-07 15:38:18.928246761 +0000 UTC m=+2247.744512837" Jul 07 15:38:19.751041 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:19.751006 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:19.755822 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:19.755793 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:19.904712 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:19.904686 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:19.905759 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:19.905742 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:21.714052 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.714022 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:38:21.810611 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.810531 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-workload-certs\") pod \"c3209d0e-fc74-414e-8ffa-190078566e1e\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " Jul 07 15:38:21.810611 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.810564 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-credential-socket\") pod \"c3209d0e-fc74-414e-8ffa-190078566e1e\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " Jul 07 15:38:21.810611 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.810584 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-data\") pod \"c3209d0e-fc74-414e-8ffa-190078566e1e\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " Jul 07 15:38:21.810611 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.810604 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-token\") pod \"c3209d0e-fc74-414e-8ffa-190078566e1e\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " Jul 07 15:38:21.810941 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.810645 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hp2sq\" (UniqueName: \"kubernetes.io/projected/c3209d0e-fc74-414e-8ffa-190078566e1e-kube-api-access-hp2sq\") pod \"c3209d0e-fc74-414e-8ffa-190078566e1e\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " Jul 07 15:38:21.810941 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.810675 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-workload-socket\") pod \"c3209d0e-fc74-414e-8ffa-190078566e1e\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " Jul 07 15:38:21.810941 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.810884 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c3209d0e-fc74-414e-8ffa-190078566e1e" (UID: "c3209d0e-fc74-414e-8ffa-190078566e1e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:38:21.810941 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.810907 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c3209d0e-fc74-414e-8ffa-190078566e1e" (UID: "c3209d0e-fc74-414e-8ffa-190078566e1e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:38:21.811157 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.810960 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-workload-certs\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:38:21.811157 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.811021 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c3209d0e-fc74-414e-8ffa-190078566e1e" (UID: "c3209d0e-fc74-414e-8ffa-190078566e1e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:38:21.811470 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.811449 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-data" (OuterVolumeSpecName: "istio-data") pod "c3209d0e-fc74-414e-8ffa-190078566e1e" (UID: "c3209d0e-fc74-414e-8ffa-190078566e1e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:38:21.812920 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.812899 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-token" (OuterVolumeSpecName: "istio-token") pod "c3209d0e-fc74-414e-8ffa-190078566e1e" (UID: "c3209d0e-fc74-414e-8ffa-190078566e1e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:38:21.813027 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.813004 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3209d0e-fc74-414e-8ffa-190078566e1e-kube-api-access-hp2sq" (OuterVolumeSpecName: "kube-api-access-hp2sq") pod "c3209d0e-fc74-414e-8ffa-190078566e1e" (UID: "c3209d0e-fc74-414e-8ffa-190078566e1e"). InnerVolumeSpecName "kube-api-access-hp2sq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:38:21.911224 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.911192 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-podinfo\") pod \"c3209d0e-fc74-414e-8ffa-190078566e1e\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " Jul 07 15:38:21.911431 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.911291 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3209d0e-fc74-414e-8ffa-190078566e1e-istiod-ca-cert\") pod \"c3209d0e-fc74-414e-8ffa-190078566e1e\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " Jul 07 15:38:21.911431 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.911413 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-envoy\") pod \"c3209d0e-fc74-414e-8ffa-190078566e1e\" (UID: \"c3209d0e-fc74-414e-8ffa-190078566e1e\") " Jul 07 15:38:21.911674 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.911642 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3209d0e-fc74-414e-8ffa-190078566e1e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c3209d0e-fc74-414e-8ffa-190078566e1e" (UID: "c3209d0e-fc74-414e-8ffa-190078566e1e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:38:21.911828 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.911714 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-workload-socket\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:38:21.911828 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.911732 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-credential-socket\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:38:21.911828 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.911745 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-data\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:38:21.911828 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.911757 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-token\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:38:21.911828 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.911769 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hp2sq\" (UniqueName: \"kubernetes.io/projected/c3209d0e-fc74-414e-8ffa-190078566e1e-kube-api-access-hp2sq\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:38:21.913392 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.913369 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c3209d0e-fc74-414e-8ffa-190078566e1e" (UID: "c3209d0e-fc74-414e-8ffa-190078566e1e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 15:38:21.913488 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.913457 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c3209d0e-fc74-414e-8ffa-190078566e1e" (UID: "c3209d0e-fc74-414e-8ffa-190078566e1e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:38:21.917705 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.917682 2556 generic.go:356] "Generic (PLEG): container finished" podID="c3209d0e-fc74-414e-8ffa-190078566e1e" containerID="7e4e56479883537de436dcd6eed7c1d8002c69a1d083828b33bb1fbbf52b573a" exitCode=0 Jul 07 15:38:21.917813 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.917733 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" Jul 07 15:38:21.917813 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.917760 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" event={"ID":"c3209d0e-fc74-414e-8ffa-190078566e1e","Type":"ContainerDied","Data":"7e4e56479883537de436dcd6eed7c1d8002c69a1d083828b33bb1fbbf52b573a"} Jul 07 15:38:21.917924 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.917822 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds" event={"ID":"c3209d0e-fc74-414e-8ffa-190078566e1e","Type":"ContainerDied","Data":"dce3b0ac6449aaff71ca21566318b6968658b7d3056903defc00c7a14aefc63c"} Jul 07 15:38:21.917924 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.917846 2556 scope.go:117] "RemoveContainer" containerID="7e4e56479883537de436dcd6eed7c1d8002c69a1d083828b33bb1fbbf52b573a" Jul 07 15:38:21.927460 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.927441 2556 scope.go:117] "RemoveContainer" containerID="4ddf2988c9a7099693ec686d4422195fd55c6566cf420140baf6628dfaead1f7" Jul 07 15:38:21.935400 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.935384 2556 scope.go:117] "RemoveContainer" containerID="7e4e56479883537de436dcd6eed7c1d8002c69a1d083828b33bb1fbbf52b573a" Jul 07 15:38:21.935670 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:38:21.935651 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e4e56479883537de436dcd6eed7c1d8002c69a1d083828b33bb1fbbf52b573a\": container with ID starting with 7e4e56479883537de436dcd6eed7c1d8002c69a1d083828b33bb1fbbf52b573a not found: ID does not exist" containerID="7e4e56479883537de436dcd6eed7c1d8002c69a1d083828b33bb1fbbf52b573a" Jul 07 15:38:21.935716 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.935682 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e4e56479883537de436dcd6eed7c1d8002c69a1d083828b33bb1fbbf52b573a"} err="failed to get container status \"7e4e56479883537de436dcd6eed7c1d8002c69a1d083828b33bb1fbbf52b573a\": rpc error: code = NotFound desc = could not find container \"7e4e56479883537de436dcd6eed7c1d8002c69a1d083828b33bb1fbbf52b573a\": container with ID starting with 7e4e56479883537de436dcd6eed7c1d8002c69a1d083828b33bb1fbbf52b573a not found: ID does not exist" Jul 07 15:38:21.935716 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.935699 2556 scope.go:117] "RemoveContainer" containerID="4ddf2988c9a7099693ec686d4422195fd55c6566cf420140baf6628dfaead1f7" Jul 07 15:38:21.936043 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:38:21.936024 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ddf2988c9a7099693ec686d4422195fd55c6566cf420140baf6628dfaead1f7\": container with ID starting with 4ddf2988c9a7099693ec686d4422195fd55c6566cf420140baf6628dfaead1f7 not found: ID does not exist" containerID="4ddf2988c9a7099693ec686d4422195fd55c6566cf420140baf6628dfaead1f7" Jul 07 15:38:21.936117 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.936054 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ddf2988c9a7099693ec686d4422195fd55c6566cf420140baf6628dfaead1f7"} err="failed to get container status \"4ddf2988c9a7099693ec686d4422195fd55c6566cf420140baf6628dfaead1f7\": rpc error: code = NotFound desc = could not find container \"4ddf2988c9a7099693ec686d4422195fd55c6566cf420140baf6628dfaead1f7\": container with ID starting with 4ddf2988c9a7099693ec686d4422195fd55c6566cf420140baf6628dfaead1f7 not found: ID does not exist" Jul 07 15:38:21.947755 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.947722 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds"] Jul 07 15:38:21.950582 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:21.950554 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-83bd55-openshift-default-5b646455d7-8rbds"] Jul 07 15:38:22.012449 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:22.012417 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-envoy\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:38:22.012449 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:22.012447 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3209d0e-fc74-414e-8ffa-190078566e1e-istio-podinfo\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:38:22.012679 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:22.012460 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3209d0e-fc74-414e-8ffa-190078566e1e-istiod-ca-cert\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:38:23.788630 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:23.788596 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3209d0e-fc74-414e-8ffa-190078566e1e" path="/var/lib/kubelet/pods/c3209d0e-fc74-414e-8ffa-190078566e1e/volumes" Jul 07 15:38:26.177892 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.160554 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl"] Jul 07 15:38:26.177892 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.161408 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3209d0e-fc74-414e-8ffa-190078566e1e" containerName="istio-proxy" Jul 07 15:38:26.177892 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.161428 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3209d0e-fc74-414e-8ffa-190078566e1e" containerName="istio-proxy" Jul 07 15:38:26.177892 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.161477 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3209d0e-fc74-414e-8ffa-190078566e1e" containerName="istio-proxy" Jul 07 15:38:26.177892 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.161485 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3209d0e-fc74-414e-8ffa-190078566e1e" containerName="istio-proxy" Jul 07 15:38:26.177892 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.161665 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3209d0e-fc74-414e-8ffa-190078566e1e" containerName="istio-proxy" Jul 07 15:38:26.177892 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.161679 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3209d0e-fc74-414e-8ffa-190078566e1e" containerName="istio-proxy" Jul 07 15:38:26.182332 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.182290 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.187309 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.187274 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-c9c5ee-openshift-default-dockercfg-7jtkm\"" Jul 07 15:38:26.196012 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.192189 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl"] Jul 07 15:38:26.251518 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.251479 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc082940-85d1-4134-88a5-1e3262d415e8-istio-podinfo\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.251687 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.251526 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-workload-certs\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.251687 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.251624 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-istio-data\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.251687 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.251661 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-credential-socket\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.251687 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.251687 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wflks\" (UniqueName: \"kubernetes.io/projected/dc082940-85d1-4134-88a5-1e3262d415e8-kube-api-access-wflks\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.251830 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.251707 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-workload-socket\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.251830 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.251732 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc082940-85d1-4134-88a5-1e3262d415e8-istiod-ca-cert\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.251925 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.251817 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-istio-envoy\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.251925 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.251862 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc082940-85d1-4134-88a5-1e3262d415e8-istio-token\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.353136 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.353096 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-istio-data\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.353310 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.353160 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-credential-socket\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.353310 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.353193 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wflks\" (UniqueName: \"kubernetes.io/projected/dc082940-85d1-4134-88a5-1e3262d415e8-kube-api-access-wflks\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.353310 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.353218 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-workload-socket\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.353310 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.353270 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc082940-85d1-4134-88a5-1e3262d415e8-istiod-ca-cert\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.353533 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.353333 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-istio-envoy\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.353533 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.353371 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc082940-85d1-4134-88a5-1e3262d415e8-istio-token\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.353533 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.353399 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc082940-85d1-4134-88a5-1e3262d415e8-istio-podinfo\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.353533 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.353423 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-workload-certs\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.353770 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.353582 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-istio-data\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.353770 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.353624 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-workload-socket\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.353901 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.353788 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-workload-certs\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.353970 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.353828 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-credential-socket\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.354041 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.354027 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc082940-85d1-4134-88a5-1e3262d415e8-istiod-ca-cert\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.356363 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.356336 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-istio-envoy\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.356658 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.356633 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc082940-85d1-4134-88a5-1e3262d415e8-istio-podinfo\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.362003 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.361976 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc082940-85d1-4134-88a5-1e3262d415e8-istio-token\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.362364 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.362346 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wflks\" (UniqueName: \"kubernetes.io/projected/dc082940-85d1-4134-88a5-1e3262d415e8-kube-api-access-wflks\") pod \"e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.504016 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.503917 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:26.683892 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.679505 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl"] Jul 07 15:38:26.694932 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:38:26.694902 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc082940_85d1_4134_88a5_1e3262d415e8.slice/crio-053052ba66ba9cf4e219effa0c78a2519a15bb1f6d32680a3223fcf922812829 WatchSource:0}: Error finding container 053052ba66ba9cf4e219effa0c78a2519a15bb1f6d32680a3223fcf922812829: Status 404 returned error can't find the container with id 053052ba66ba9cf4e219effa0c78a2519a15bb1f6d32680a3223fcf922812829 Jul 07 15:38:26.697392 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.697357 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:38:26.697476 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.697428 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:38:26.697476 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.697455 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:38:26.945670 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.945628 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" event={"ID":"dc082940-85d1-4134-88a5-1e3262d415e8","Type":"ContainerStarted","Data":"5489bc69c2e66f031a8fb5f8c7fe1b990683d72a70245e8d819c4326481e51d1"} Jul 07 15:38:26.945821 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.945689 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" event={"ID":"dc082940-85d1-4134-88a5-1e3262d415e8","Type":"ContainerStarted","Data":"053052ba66ba9cf4e219effa0c78a2519a15bb1f6d32680a3223fcf922812829"} Jul 07 15:38:26.969593 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:26.969531 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" podStartSLOduration=0.969510435 podStartE2EDuration="969.510435ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 15:38:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:38:26.966029418 +0000 UTC m=+2255.782295482" watchObservedRunningTime="2026-07-07 15:38:26.969510435 +0000 UTC m=+2255.785776478" Jul 07 15:38:27.504277 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:27.504239 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:27.509530 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:27.509505 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:27.950888 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:27.950834 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:27.951945 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:27.951925 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:33.983689 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:33.983652 2556 generic.go:356] "Generic (PLEG): container finished" podID="11d693e8-dd5f-4fd0-8192-5410a3539741" containerID="0d8e97ad091050ca70cf72daaa86998c05f1e5b4670f19780eb1348d85c0dc10" exitCode=0 Jul 07 15:38:33.984134 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:33.983723 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" event={"ID":"11d693e8-dd5f-4fd0-8192-5410a3539741","Type":"ContainerDied","Data":"0d8e97ad091050ca70cf72daaa86998c05f1e5b4670f19780eb1348d85c0dc10"} Jul 07 15:38:33.984194 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:33.984180 2556 scope.go:117] "RemoveContainer" containerID="0d8e97ad091050ca70cf72daaa86998c05f1e5b4670f19780eb1348d85c0dc10" Jul 07 15:38:33.985213 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:33.985186 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:38:33.985275 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:33.985252 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:38:33.985311 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:33.985286 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:38:34.905144 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:34.905100 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:34.993906 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:34.993848 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" event={"ID":"11d693e8-dd5f-4fd0-8192-5410a3539741","Type":"ContainerStarted","Data":"8004aecd16194675386c90e690bbb7868873e8a89687e486bed611bf98fe5e2e"} Jul 07 15:38:35.751438 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:35.751396 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:35.752597 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:35.752573 2556 patch_prober.go:28] interesting pod/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 07 15:38:35.752667 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:35.752635 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" podUID="11d693e8-dd5f-4fd0-8192-5410a3539741" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 07 15:38:36.751732 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:36.751702 2556 patch_prober.go:28] interesting pod/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 07 15:38:36.752242 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:36.752205 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" podUID="11d693e8-dd5f-4fd0-8192-5410a3539741" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 07 15:38:37.751242 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:37.751208 2556 patch_prober.go:28] interesting pod/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 07 15:38:37.751586 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:37.751557 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" podUID="11d693e8-dd5f-4fd0-8192-5410a3539741" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 07 15:38:38.751605 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:38.751565 2556 patch_prober.go:28] interesting pod/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 07 15:38:38.752052 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:38.751624 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" podUID="11d693e8-dd5f-4fd0-8192-5410a3539741" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 07 15:38:39.755091 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:39.755062 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:39.755540 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:39.755401 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:39.756288 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:39.756269 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:38:42.952500 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:42.952459 2556 patch_prober.go:28] interesting pod/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 07 15:38:42.952949 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:42.952537 2556 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" podUID="dc082940-85d1-4134-88a5-1e3262d415e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 07 15:38:43.049681 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:43.049648 2556 generic.go:356] "Generic (PLEG): container finished" podID="dc082940-85d1-4134-88a5-1e3262d415e8" containerID="5489bc69c2e66f031a8fb5f8c7fe1b990683d72a70245e8d819c4326481e51d1" exitCode=0 Jul 07 15:38:43.049855 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:43.049721 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" event={"ID":"dc082940-85d1-4134-88a5-1e3262d415e8","Type":"ContainerDied","Data":"5489bc69c2e66f031a8fb5f8c7fe1b990683d72a70245e8d819c4326481e51d1"} Jul 07 15:38:43.050153 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:43.050138 2556 scope.go:117] "RemoveContainer" containerID="5489bc69c2e66f031a8fb5f8c7fe1b990683d72a70245e8d819c4326481e51d1" Jul 07 15:38:43.051124 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:43.051099 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:38:43.051193 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:43.051167 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:38:43.051230 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:43.051197 2556 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 15:38:44.057781 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:44.057733 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" event={"ID":"dc082940-85d1-4134-88a5-1e3262d415e8","Type":"ContainerStarted","Data":"cc673741e739545526145bf3cf4038f3788b8216a733e5b13ab43f366ca3e888"} Jul 07 15:38:44.504979 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:44.504940 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:44.505213 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:44.505190 2556 patch_prober.go:28] interesting pod/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 07 15:38:44.505290 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:44.505239 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" podUID="dc082940-85d1-4134-88a5-1e3262d415e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 07 15:38:45.504502 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:45.504466 2556 patch_prober.go:28] interesting pod/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 07 15:38:45.504951 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:45.504594 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" podUID="dc082940-85d1-4134-88a5-1e3262d415e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 07 15:38:46.504967 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:46.504932 2556 patch_prober.go:28] interesting pod/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 07 15:38:46.505446 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:46.505005 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" podUID="dc082940-85d1-4134-88a5-1e3262d415e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 07 15:38:47.505315 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:47.505275 2556 patch_prober.go:28] interesting pod/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 07 15:38:47.505703 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:47.505355 2556 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" podUID="dc082940-85d1-4134-88a5-1e3262d415e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 07 15:38:48.508831 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:48.508802 2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:48.509219 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:48.509090 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:38:48.509888 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:38:48.509855 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:40:51.807500 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:40:51.807470 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/1.log" Jul 07 15:40:51.811283 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:40:51.811258 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/1.log" Jul 07 15:44:47.082718 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:47.082681 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v"] Jul 07 15:44:47.083245 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:47.082946 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" podUID="11d693e8-dd5f-4fd0-8192-5410a3539741" containerName="istio-proxy" containerID="cri-o://8004aecd16194675386c90e690bbb7868873e8a89687e486bed611bf98fe5e2e" gracePeriod=30 Jul 07 15:44:52.335407 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.335370 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:44:52.502835 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.502745 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-data\") pod \"11d693e8-dd5f-4fd0-8192-5410a3539741\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " Jul 07 15:44:52.502835 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.502803 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11d693e8-dd5f-4fd0-8192-5410a3539741-istiod-ca-cert\") pod \"11d693e8-dd5f-4fd0-8192-5410a3539741\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " Jul 07 15:44:52.502835 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.502829 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-credential-socket\") pod \"11d693e8-dd5f-4fd0-8192-5410a3539741\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " Jul 07 15:44:52.503180 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.502854 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-workload-certs\") pod \"11d693e8-dd5f-4fd0-8192-5410a3539741\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " Jul 07 15:44:52.503180 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.503025 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-envoy\") pod \"11d693e8-dd5f-4fd0-8192-5410a3539741\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " Jul 07 15:44:52.503180 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.503098 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-token\") pod \"11d693e8-dd5f-4fd0-8192-5410a3539741\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " Jul 07 15:44:52.503180 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.503146 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-workload-socket\") pod \"11d693e8-dd5f-4fd0-8192-5410a3539741\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " Jul 07 15:44:52.503180 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.503170 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "11d693e8-dd5f-4fd0-8192-5410a3539741" (UID: "11d693e8-dd5f-4fd0-8192-5410a3539741"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:44:52.503464 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.503184 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82bh9\" (UniqueName: \"kubernetes.io/projected/11d693e8-dd5f-4fd0-8192-5410a3539741-kube-api-access-82bh9\") pod \"11d693e8-dd5f-4fd0-8192-5410a3539741\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " Jul 07 15:44:52.503464 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.503183 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "11d693e8-dd5f-4fd0-8192-5410a3539741" (UID: "11d693e8-dd5f-4fd0-8192-5410a3539741"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:44:52.503464 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.503282 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11d693e8-dd5f-4fd0-8192-5410a3539741-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "11d693e8-dd5f-4fd0-8192-5410a3539741" (UID: "11d693e8-dd5f-4fd0-8192-5410a3539741"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:44:52.503464 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.503302 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-podinfo\") pod \"11d693e8-dd5f-4fd0-8192-5410a3539741\" (UID: \"11d693e8-dd5f-4fd0-8192-5410a3539741\") " Jul 07 15:44:52.503464 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.503398 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "11d693e8-dd5f-4fd0-8192-5410a3539741" (UID: "11d693e8-dd5f-4fd0-8192-5410a3539741"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:44:52.503696 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.503661 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11d693e8-dd5f-4fd0-8192-5410a3539741-istiod-ca-cert\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:44:52.503696 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.503681 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-credential-socket\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:44:52.503785 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.503694 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-workload-certs\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:44:52.503785 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.503708 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-workload-socket\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:44:52.503898 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.503820 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-data" (OuterVolumeSpecName: "istio-data") pod "11d693e8-dd5f-4fd0-8192-5410a3539741" (UID: "11d693e8-dd5f-4fd0-8192-5410a3539741"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:44:52.505368 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.505343 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "11d693e8-dd5f-4fd0-8192-5410a3539741" (UID: "11d693e8-dd5f-4fd0-8192-5410a3539741"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:44:52.505486 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.505384 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-token" (OuterVolumeSpecName: "istio-token") pod "11d693e8-dd5f-4fd0-8192-5410a3539741" (UID: "11d693e8-dd5f-4fd0-8192-5410a3539741"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:44:52.505638 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.505616 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11d693e8-dd5f-4fd0-8192-5410a3539741-kube-api-access-82bh9" (OuterVolumeSpecName: "kube-api-access-82bh9") pod "11d693e8-dd5f-4fd0-8192-5410a3539741" (UID: "11d693e8-dd5f-4fd0-8192-5410a3539741"). InnerVolumeSpecName "kube-api-access-82bh9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:44:52.505638 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.505622 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "11d693e8-dd5f-4fd0-8192-5410a3539741" (UID: "11d693e8-dd5f-4fd0-8192-5410a3539741"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 15:44:52.605041 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.605003 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-envoy\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:44:52.605041 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.605036 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-token\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:44:52.605256 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.605051 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82bh9\" (UniqueName: \"kubernetes.io/projected/11d693e8-dd5f-4fd0-8192-5410a3539741-kube-api-access-82bh9\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:44:52.605256 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.605067 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-podinfo\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:44:52.605256 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.605079 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11d693e8-dd5f-4fd0-8192-5410a3539741-istio-data\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:44:52.823218 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.823120 2556 generic.go:356] "Generic (PLEG): container finished" podID="11d693e8-dd5f-4fd0-8192-5410a3539741" containerID="8004aecd16194675386c90e690bbb7868873e8a89687e486bed611bf98fe5e2e" exitCode=0 Jul 07 15:44:52.823218 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.823172 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" event={"ID":"11d693e8-dd5f-4fd0-8192-5410a3539741","Type":"ContainerDied","Data":"8004aecd16194675386c90e690bbb7868873e8a89687e486bed611bf98fe5e2e"} Jul 07 15:44:52.823218 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.823182 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" Jul 07 15:44:52.823483 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.823226 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v" event={"ID":"11d693e8-dd5f-4fd0-8192-5410a3539741","Type":"ContainerDied","Data":"1f859acae38beb1b601d505b22190a027e5a0471bf181cba04a3d470daac0b8e"} Jul 07 15:44:52.823483 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.823250 2556 scope.go:117] "RemoveContainer" containerID="8004aecd16194675386c90e690bbb7868873e8a89687e486bed611bf98fe5e2e" Jul 07 15:44:52.832500 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.832483 2556 scope.go:117] "RemoveContainer" containerID="0d8e97ad091050ca70cf72daaa86998c05f1e5b4670f19780eb1348d85c0dc10" Jul 07 15:44:52.840355 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.840336 2556 scope.go:117] "RemoveContainer" containerID="8004aecd16194675386c90e690bbb7868873e8a89687e486bed611bf98fe5e2e" Jul 07 15:44:52.840617 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:44:52.840591 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8004aecd16194675386c90e690bbb7868873e8a89687e486bed611bf98fe5e2e\": container with ID starting with 8004aecd16194675386c90e690bbb7868873e8a89687e486bed611bf98fe5e2e not found: ID does not exist" containerID="8004aecd16194675386c90e690bbb7868873e8a89687e486bed611bf98fe5e2e" Jul 07 15:44:52.840670 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.840632 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8004aecd16194675386c90e690bbb7868873e8a89687e486bed611bf98fe5e2e"} err="failed to get container status \"8004aecd16194675386c90e690bbb7868873e8a89687e486bed611bf98fe5e2e\": rpc error: code = NotFound desc = could not find container \"8004aecd16194675386c90e690bbb7868873e8a89687e486bed611bf98fe5e2e\": container with ID starting with 8004aecd16194675386c90e690bbb7868873e8a89687e486bed611bf98fe5e2e not found: ID does not exist" Jul 07 15:44:52.840670 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.840650 2556 scope.go:117] "RemoveContainer" containerID="0d8e97ad091050ca70cf72daaa86998c05f1e5b4670f19780eb1348d85c0dc10" Jul 07 15:44:52.840853 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:44:52.840838 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d8e97ad091050ca70cf72daaa86998c05f1e5b4670f19780eb1348d85c0dc10\": container with ID starting with 0d8e97ad091050ca70cf72daaa86998c05f1e5b4670f19780eb1348d85c0dc10 not found: ID does not exist" containerID="0d8e97ad091050ca70cf72daaa86998c05f1e5b4670f19780eb1348d85c0dc10" Jul 07 15:44:52.840925 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.840857 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d8e97ad091050ca70cf72daaa86998c05f1e5b4670f19780eb1348d85c0dc10"} err="failed to get container status \"0d8e97ad091050ca70cf72daaa86998c05f1e5b4670f19780eb1348d85c0dc10\": rpc error: code = NotFound desc = could not find container \"0d8e97ad091050ca70cf72daaa86998c05f1e5b4670f19780eb1348d85c0dc10\": container with ID starting with 0d8e97ad091050ca70cf72daaa86998c05f1e5b4670f19780eb1348d85c0dc10 not found: ID does not exist" Jul 07 15:44:52.846752 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.846728 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v"] Jul 07 15:44:52.852006 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:52.851985 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-f09258-openshift-default-856dcf9c75-r6d2v"] Jul 07 15:44:53.788625 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:53.788590 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11d693e8-dd5f-4fd0-8192-5410a3539741" path="/var/lib/kubelet/pods/11d693e8-dd5f-4fd0-8192-5410a3539741/volumes" Jul 07 15:44:54.103489 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:54.103456 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl"] Jul 07 15:44:54.103735 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:54.103707 2556 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" podUID="dc082940-85d1-4134-88a5-1e3262d415e8" containerName="istio-proxy" containerID="cri-o://cc673741e739545526145bf3cf4038f3788b8216a733e5b13ab43f366ca3e888" gracePeriod=30 Jul 07 15:44:59.360617 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.360587 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:44:59.367726 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.367654 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc082940-85d1-4134-88a5-1e3262d415e8-istiod-ca-cert\") pod \"dc082940-85d1-4134-88a5-1e3262d415e8\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " Jul 07 15:44:59.367726 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.367707 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-workload-certs\") pod \"dc082940-85d1-4134-88a5-1e3262d415e8\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " Jul 07 15:44:59.367935 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.367738 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc082940-85d1-4134-88a5-1e3262d415e8-istio-token\") pod \"dc082940-85d1-4134-88a5-1e3262d415e8\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " Jul 07 15:44:59.367935 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.367765 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc082940-85d1-4134-88a5-1e3262d415e8-istio-podinfo\") pod \"dc082940-85d1-4134-88a5-1e3262d415e8\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " Jul 07 15:44:59.367935 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.367796 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wflks\" (UniqueName: \"kubernetes.io/projected/dc082940-85d1-4134-88a5-1e3262d415e8-kube-api-access-wflks\") pod \"dc082940-85d1-4134-88a5-1e3262d415e8\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " Jul 07 15:44:59.367935 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.367839 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-istio-data\") pod \"dc082940-85d1-4134-88a5-1e3262d415e8\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " Jul 07 15:44:59.367935 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.367928 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-istio-envoy\") pod \"dc082940-85d1-4134-88a5-1e3262d415e8\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " Jul 07 15:44:59.368188 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.367978 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-workload-socket\") pod \"dc082940-85d1-4134-88a5-1e3262d415e8\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " Jul 07 15:44:59.368188 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.367996 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dc082940-85d1-4134-88a5-1e3262d415e8" (UID: "dc082940-85d1-4134-88a5-1e3262d415e8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:44:59.368188 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.368078 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-credential-socket\") pod \"dc082940-85d1-4134-88a5-1e3262d415e8\" (UID: \"dc082940-85d1-4134-88a5-1e3262d415e8\") " Jul 07 15:44:59.368188 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.368116 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc082940-85d1-4134-88a5-1e3262d415e8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dc082940-85d1-4134-88a5-1e3262d415e8" (UID: "dc082940-85d1-4134-88a5-1e3262d415e8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 15:44:59.368401 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.368359 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dc082940-85d1-4134-88a5-1e3262d415e8" (UID: "dc082940-85d1-4134-88a5-1e3262d415e8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:44:59.368494 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.368472 2556 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-credential-socket\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:44:59.368556 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.368500 2556 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dc082940-85d1-4134-88a5-1e3262d415e8-istiod-ca-cert\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:44:59.368556 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.368513 2556 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-workload-certs\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:44:59.368556 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.368535 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dc082940-85d1-4134-88a5-1e3262d415e8" (UID: "dc082940-85d1-4134-88a5-1e3262d415e8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:44:59.369053 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.369024 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-istio-data" (OuterVolumeSpecName: "istio-data") pod "dc082940-85d1-4134-88a5-1e3262d415e8" (UID: "dc082940-85d1-4134-88a5-1e3262d415e8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:44:59.370215 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.370185 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dc082940-85d1-4134-88a5-1e3262d415e8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dc082940-85d1-4134-88a5-1e3262d415e8" (UID: "dc082940-85d1-4134-88a5-1e3262d415e8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 15:44:59.370479 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.370457 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc082940-85d1-4134-88a5-1e3262d415e8-kube-api-access-wflks" (OuterVolumeSpecName: "kube-api-access-wflks") pod "dc082940-85d1-4134-88a5-1e3262d415e8" (UID: "dc082940-85d1-4134-88a5-1e3262d415e8"). InnerVolumeSpecName "kube-api-access-wflks". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:44:59.370552 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.370491 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dc082940-85d1-4134-88a5-1e3262d415e8" (UID: "dc082940-85d1-4134-88a5-1e3262d415e8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 15:44:59.370552 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.370498 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc082940-85d1-4134-88a5-1e3262d415e8-istio-token" (OuterVolumeSpecName: "istio-token") pod "dc082940-85d1-4134-88a5-1e3262d415e8" (UID: "dc082940-85d1-4134-88a5-1e3262d415e8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:44:59.470010 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.469971 2556 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-workload-socket\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:44:59.470010 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.470003 2556 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dc082940-85d1-4134-88a5-1e3262d415e8-istio-token\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:44:59.470010 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.470011 2556 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dc082940-85d1-4134-88a5-1e3262d415e8-istio-podinfo\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:44:59.470247 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.470027 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wflks\" (UniqueName: \"kubernetes.io/projected/dc082940-85d1-4134-88a5-1e3262d415e8-kube-api-access-wflks\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:44:59.470247 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.470037 2556 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-istio-data\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:44:59.470247 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.470045 2556 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dc082940-85d1-4134-88a5-1e3262d415e8-istio-envoy\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:44:59.859604 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.859566 2556 generic.go:356] "Generic (PLEG): container finished" podID="dc082940-85d1-4134-88a5-1e3262d415e8" containerID="cc673741e739545526145bf3cf4038f3788b8216a733e5b13ab43f366ca3e888" exitCode=0 Jul 07 15:44:59.859790 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.859618 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" Jul 07 15:44:59.859790 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.859645 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" event={"ID":"dc082940-85d1-4134-88a5-1e3262d415e8","Type":"ContainerDied","Data":"cc673741e739545526145bf3cf4038f3788b8216a733e5b13ab43f366ca3e888"} Jul 07 15:44:59.859790 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.859698 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl" event={"ID":"dc082940-85d1-4134-88a5-1e3262d415e8","Type":"ContainerDied","Data":"053052ba66ba9cf4e219effa0c78a2519a15bb1f6d32680a3223fcf922812829"} Jul 07 15:44:59.859790 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.859727 2556 scope.go:117] "RemoveContainer" containerID="cc673741e739545526145bf3cf4038f3788b8216a733e5b13ab43f366ca3e888" Jul 07 15:44:59.868588 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.868571 2556 scope.go:117] "RemoveContainer" containerID="5489bc69c2e66f031a8fb5f8c7fe1b990683d72a70245e8d819c4326481e51d1" Jul 07 15:44:59.877653 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.877607 2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl"] Jul 07 15:44:59.877956 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.877650 2556 scope.go:117] "RemoveContainer" containerID="cc673741e739545526145bf3cf4038f3788b8216a733e5b13ab43f366ca3e888" Jul 07 15:44:59.878030 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:44:59.877948 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc673741e739545526145bf3cf4038f3788b8216a733e5b13ab43f366ca3e888\": container with ID starting with cc673741e739545526145bf3cf4038f3788b8216a733e5b13ab43f366ca3e888 not found: ID does not exist" containerID="cc673741e739545526145bf3cf4038f3788b8216a733e5b13ab43f366ca3e888" Jul 07 15:44:59.878030 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.877975 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc673741e739545526145bf3cf4038f3788b8216a733e5b13ab43f366ca3e888"} err="failed to get container status \"cc673741e739545526145bf3cf4038f3788b8216a733e5b13ab43f366ca3e888\": rpc error: code = NotFound desc = could not find container \"cc673741e739545526145bf3cf4038f3788b8216a733e5b13ab43f366ca3e888\": container with ID starting with cc673741e739545526145bf3cf4038f3788b8216a733e5b13ab43f366ca3e888 not found: ID does not exist" Jul 07 15:44:59.878030 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.878026 2556 scope.go:117] "RemoveContainer" containerID="5489bc69c2e66f031a8fb5f8c7fe1b990683d72a70245e8d819c4326481e51d1" Jul 07 15:44:59.878429 ip-10-0-130-39 kubenswrapper[2556]: E0707 15:44:59.878369 2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5489bc69c2e66f031a8fb5f8c7fe1b990683d72a70245e8d819c4326481e51d1\": container with ID starting with 5489bc69c2e66f031a8fb5f8c7fe1b990683d72a70245e8d819c4326481e51d1 not found: ID does not exist" containerID="5489bc69c2e66f031a8fb5f8c7fe1b990683d72a70245e8d819c4326481e51d1" Jul 07 15:44:59.878541 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.878498 2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5489bc69c2e66f031a8fb5f8c7fe1b990683d72a70245e8d819c4326481e51d1"} err="failed to get container status \"5489bc69c2e66f031a8fb5f8c7fe1b990683d72a70245e8d819c4326481e51d1\": rpc error: code = NotFound desc = could not find container \"5489bc69c2e66f031a8fb5f8c7fe1b990683d72a70245e8d819c4326481e51d1\": container with ID starting with 5489bc69c2e66f031a8fb5f8c7fe1b990683d72a70245e8d819c4326481e51d1 not found: ID does not exist" Jul 07 15:44:59.880850 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:44:59.880829 2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-c9c5ee-openshift-default-66b79c8747-78cxl"] Jul 07 15:45:00.140507 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:00.140471 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723985-74vdm"] Jul 07 15:45:00.140832 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:00.140819 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11d693e8-dd5f-4fd0-8192-5410a3539741" containerName="istio-proxy" Jul 07 15:45:00.140887 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:00.140835 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="11d693e8-dd5f-4fd0-8192-5410a3539741" containerName="istio-proxy" Jul 07 15:45:00.140887 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:00.140849 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc082940-85d1-4134-88a5-1e3262d415e8" containerName="istio-proxy" Jul 07 15:45:00.140887 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:00.140854 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc082940-85d1-4134-88a5-1e3262d415e8" containerName="istio-proxy" Jul 07 15:45:00.140887 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:00.140881 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc082940-85d1-4134-88a5-1e3262d415e8" containerName="istio-proxy" Jul 07 15:45:00.140887 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:00.140887 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc082940-85d1-4134-88a5-1e3262d415e8" containerName="istio-proxy" Jul 07 15:45:00.141058 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:00.140940 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="11d693e8-dd5f-4fd0-8192-5410a3539741" containerName="istio-proxy" Jul 07 15:45:00.141058 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:00.140947 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="11d693e8-dd5f-4fd0-8192-5410a3539741" containerName="istio-proxy" Jul 07 15:45:00.141058 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:00.140954 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc082940-85d1-4134-88a5-1e3262d415e8" containerName="istio-proxy" Jul 07 15:45:00.141058 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:00.140961 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc082940-85d1-4134-88a5-1e3262d415e8" containerName="istio-proxy" Jul 07 15:45:00.144329 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:00.144306 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723985-74vdm" Jul 07 15:45:00.146514 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:00.146499 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-td6sd\"" Jul 07 15:45:00.150805 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:00.150756 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723985-74vdm"] Jul 07 15:45:00.176681 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:00.176647 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt87k\" (UniqueName: \"kubernetes.io/projected/e7df1e71-db4a-4023-b1d7-29c0a4905471-kube-api-access-bt87k\") pod \"maas-api-key-cleanup-29723985-74vdm\" (UID: \"e7df1e71-db4a-4023-b1d7-29c0a4905471\") " pod="opendatahub/maas-api-key-cleanup-29723985-74vdm" Jul 07 15:45:00.223452 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:00.223416 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723955-8wgxz_3bca2b6e-9022-4c85-a582-e89a7d8f55fc/cleanup/0.log" Jul 07 15:45:00.277783 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:00.277746 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt87k\" (UniqueName: \"kubernetes.io/projected/e7df1e71-db4a-4023-b1d7-29c0a4905471-kube-api-access-bt87k\") pod \"maas-api-key-cleanup-29723985-74vdm\" (UID: \"e7df1e71-db4a-4023-b1d7-29c0a4905471\") " pod="opendatahub/maas-api-key-cleanup-29723985-74vdm" Jul 07 15:45:00.287256 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:00.287230 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt87k\" (UniqueName: \"kubernetes.io/projected/e7df1e71-db4a-4023-b1d7-29c0a4905471-kube-api-access-bt87k\") pod \"maas-api-key-cleanup-29723985-74vdm\" (UID: \"e7df1e71-db4a-4023-b1d7-29c0a4905471\") " pod="opendatahub/maas-api-key-cleanup-29723985-74vdm" Jul 07 15:45:00.346646 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:00.346615 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723970-lqpkr_2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49/cleanup/0.log" Jul 07 15:45:00.455945 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:00.455837 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723985-74vdm" Jul 07 15:45:00.794887 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:00.794834 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723985-74vdm"] Jul 07 15:45:00.797119 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:45:00.797091 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7df1e71_db4a_4023_b1d7_29c0a4905471.slice/crio-a36858950956a14ca15c01fd22b9340fd35c563ee226e588ec92e5171ee139df WatchSource:0}: Error finding container a36858950956a14ca15c01fd22b9340fd35c563ee226e588ec92e5171ee139df: Status 404 returned error can't find the container with id a36858950956a14ca15c01fd22b9340fd35c563ee226e588ec92e5171ee139df Jul 07 15:45:00.867508 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:00.867473 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723985-74vdm" event={"ID":"e7df1e71-db4a-4023-b1d7-29c0a4905471","Type":"ContainerStarted","Data":"a36858950956a14ca15c01fd22b9340fd35c563ee226e588ec92e5171ee139df"} Jul 07 15:45:01.177837 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:01.177812 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6_728e97a8-e828-46f0-bb12-6aa37b215e9a/copy-manifests/0.log" Jul 07 15:45:01.185905 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:01.185859 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6_728e97a8-e828-46f0-bb12-6aa37b215e9a/manager/0.log" Jul 07 15:45:01.788771 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:01.788741 2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc082940-85d1-4134-88a5-1e3262d415e8" path="/var/lib/kubelet/pods/dc082940-85d1-4134-88a5-1e3262d415e8/volumes" Jul 07 15:45:01.873251 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:01.873215 2556 generic.go:356] "Generic (PLEG): container finished" podID="e7df1e71-db4a-4023-b1d7-29c0a4905471" containerID="2970bc28e175f2c94aadc429ec2ab53b8da0f8b192216943dce600b9f87f0f27" exitCode=0 Jul 07 15:45:01.873413 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:01.873304 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723985-74vdm" event={"ID":"e7df1e71-db4a-4023-b1d7-29c0a4905471","Type":"ContainerDied","Data":"2970bc28e175f2c94aadc429ec2ab53b8da0f8b192216943dce600b9f87f0f27"} Jul 07 15:45:03.008217 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:03.008173 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723985-74vdm" Jul 07 15:45:03.103216 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:03.103169 2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bt87k\" (UniqueName: \"kubernetes.io/projected/e7df1e71-db4a-4023-b1d7-29c0a4905471-kube-api-access-bt87k\") pod \"e7df1e71-db4a-4023-b1d7-29c0a4905471\" (UID: \"e7df1e71-db4a-4023-b1d7-29c0a4905471\") " Jul 07 15:45:03.105263 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:03.105231 2556 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7df1e71-db4a-4023-b1d7-29c0a4905471-kube-api-access-bt87k" (OuterVolumeSpecName: "kube-api-access-bt87k") pod "e7df1e71-db4a-4023-b1d7-29c0a4905471" (UID: "e7df1e71-db4a-4023-b1d7-29c0a4905471"). InnerVolumeSpecName "kube-api-access-bt87k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 15:45:03.203889 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:03.203779 2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bt87k\" (UniqueName: \"kubernetes.io/projected/e7df1e71-db4a-4023-b1d7-29c0a4905471-kube-api-access-bt87k\") on node \"ip-10-0-130-39.ec2.internal\" DevicePath \"\"" Jul 07 15:45:03.669011 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:03.668981 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-zwm9m_2bf1b387-02dd-4ef5-9850-ab3e648c2899/manager/0.log" Jul 07 15:45:03.883740 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:03.883700 2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723985-74vdm" Jul 07 15:45:03.883740 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:03.883728 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723985-74vdm" event={"ID":"e7df1e71-db4a-4023-b1d7-29c0a4905471","Type":"ContainerDied","Data":"a36858950956a14ca15c01fd22b9340fd35c563ee226e588ec92e5171ee139df"} Jul 07 15:45:03.884000 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:03.883785 2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a36858950956a14ca15c01fd22b9340fd35c563ee226e588ec92e5171ee139df" Jul 07 15:45:04.174389 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:04.174358 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-482t6_98339dff-159d-40b8-b7f1-81492771670d/discovery/0.log" Jul 07 15:45:04.793574 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:04.793545 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-qn6gm_117c8485-434b-4a08-9a53-4883af58af7a/payload-processing/0.log" Jul 07 15:45:04.915719 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:04.915684 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c76867bd5-72zmj_0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7/router/0.log" Jul 07 15:45:05.539732 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:05.539673 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-5gjlx_765b5121-c00d-454e-b5c7-e572ec300fe0/main/0.log" Jul 07 15:45:13.165753 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:13.165717 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-n6c4h_66af1c92-bb7a-4cda-bb2f-48fa2353b00e/global-pull-secret-syncer/0.log" Jul 07 15:45:13.300395 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:13.300367 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tzhn6_ce32c7aa-dbf8-4ba2-bdd8-cfe8f4b6ca91/konnectivity-agent/0.log" Jul 07 15:45:13.353928 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:13.353900 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-39.ec2.internal_1931978687adf475d28ebf31d0441510/haproxy/0.log" Jul 07 15:45:18.403469 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:18.403373 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-zwm9m_2bf1b387-02dd-4ef5-9850-ab3e648c2899/manager/0.log" Jul 07 15:45:20.140996 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:20.140962 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-q9n5d_cc9cdf22-7b80-4c4e-941a-305d93a9ea4d/cluster-monitoring-operator/0.log" Jul 07 15:45:20.178968 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:20.178934 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-5qxsn_d10f41d4-e793-447e-a01f-c04f1872686f/kube-state-metrics/0.log" Jul 07 15:45:20.197711 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:20.197680 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-5qxsn_d10f41d4-e793-447e-a01f-c04f1872686f/kube-rbac-proxy-main/0.log" Jul 07 15:45:20.217846 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:20.217763 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-5qxsn_d10f41d4-e793-447e-a01f-c04f1872686f/kube-rbac-proxy-self/0.log" Jul 07 15:45:20.243474 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:20.243451 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-75b4cf9bdb-65swh_f6f9475c-cfa1-424b-a7be-b5601290ecb9/metrics-server/0.log" Jul 07 15:45:20.485456 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:20.485378 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-md95l_ddf9624c-3835-414e-8b3b-a3c26de812b9/node-exporter/0.log" Jul 07 15:45:20.503537 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:20.503513 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-md95l_ddf9624c-3835-414e-8b3b-a3c26de812b9/kube-rbac-proxy/0.log" Jul 07 15:45:20.523655 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:20.523614 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-md95l_ddf9624c-3835-414e-8b3b-a3c26de812b9/init-textfile/0.log" Jul 07 15:45:20.650705 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:20.650675 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7ca68156-8bab-4e82-85c4-0695c94f3f64/prometheus/0.log" Jul 07 15:45:20.674347 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:20.674320 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7ca68156-8bab-4e82-85c4-0695c94f3f64/config-reloader/0.log" Jul 07 15:45:20.692782 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:20.692759 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7ca68156-8bab-4e82-85c4-0695c94f3f64/thanos-sidecar/0.log" Jul 07 15:45:20.711768 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:20.711741 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7ca68156-8bab-4e82-85c4-0695c94f3f64/kube-rbac-proxy-web/0.log" Jul 07 15:45:20.732669 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:20.732646 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7ca68156-8bab-4e82-85c4-0695c94f3f64/kube-rbac-proxy/0.log" Jul 07 15:45:20.753168 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:20.753054 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7ca68156-8bab-4e82-85c4-0695c94f3f64/kube-rbac-proxy-thanos/0.log" Jul 07 15:45:20.787428 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:20.787406 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7ca68156-8bab-4e82-85c4-0695c94f3f64/init-config-reloader/0.log" Jul 07 15:45:20.993688 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:20.993663 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84d5487686-dxwqc_46ab3c80-fb4e-46a0-914f-a873aec066ba/thanos-query/0.log" Jul 07 15:45:21.013183 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:21.013106 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84d5487686-dxwqc_46ab3c80-fb4e-46a0-914f-a873aec066ba/kube-rbac-proxy-web/0.log" Jul 07 15:45:21.040014 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:21.039987 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84d5487686-dxwqc_46ab3c80-fb4e-46a0-914f-a873aec066ba/kube-rbac-proxy/0.log" Jul 07 15:45:21.070133 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:21.070106 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84d5487686-dxwqc_46ab3c80-fb4e-46a0-914f-a873aec066ba/prom-label-proxy/0.log" Jul 07 15:45:21.094822 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:21.094797 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84d5487686-dxwqc_46ab3c80-fb4e-46a0-914f-a873aec066ba/kube-rbac-proxy-rules/0.log" Jul 07 15:45:21.134850 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:21.134792 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-84d5487686-dxwqc_46ab3c80-fb4e-46a0-914f-a873aec066ba/kube-rbac-proxy-metrics/0.log" Jul 07 15:45:23.196946 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:23.196914 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/1.log" Jul 07 15:45:23.205841 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:23.205816 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-z4h8q_4faf5ae7-9e9a-41ac-aad2-9808d11d4c42/console-operator/2.log" Jul 07 15:45:23.767940 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:23.767906 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-jzb97_776f3b37-8cdd-4fd4-8d07-c78ced246a79/download-server/0.log" Jul 07 15:45:24.006471 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.006443 2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w"] Jul 07 15:45:24.006829 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.006815 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11d693e8-dd5f-4fd0-8192-5410a3539741" containerName="istio-proxy" Jul 07 15:45:24.006883 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.006857 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="11d693e8-dd5f-4fd0-8192-5410a3539741" containerName="istio-proxy" Jul 07 15:45:24.006929 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.006881 2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7df1e71-db4a-4023-b1d7-29c0a4905471" containerName="cleanup" Jul 07 15:45:24.006929 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.006892 2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7df1e71-db4a-4023-b1d7-29c0a4905471" containerName="cleanup" Jul 07 15:45:24.006995 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.006968 2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7df1e71-db4a-4023-b1d7-29c0a4905471" containerName="cleanup" Jul 07 15:45:24.009731 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.009708 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w" Jul 07 15:45:24.012120 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.012099 2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-d8xvs\"/\"default-dockercfg-lz587\"" Jul 07 15:45:24.012236 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.012207 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d8xvs\"/\"openshift-service-ca.crt\"" Jul 07 15:45:24.012990 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.012976 2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-d8xvs\"/\"kube-root-ca.crt\"" Jul 07 15:45:24.025370 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.025304 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w"] Jul 07 15:45:24.100434 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.100404 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b175d704-58a8-44c5-8e4c-bc83aaba9366-sys\") pod \"perf-node-gather-daemonset-swb7w\" (UID: \"b175d704-58a8-44c5-8e4c-bc83aaba9366\") " pod="openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w" Jul 07 15:45:24.100639 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.100457 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k86cm\" (UniqueName: \"kubernetes.io/projected/b175d704-58a8-44c5-8e4c-bc83aaba9366-kube-api-access-k86cm\") pod \"perf-node-gather-daemonset-swb7w\" (UID: \"b175d704-58a8-44c5-8e4c-bc83aaba9366\") " pod="openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w" Jul 07 15:45:24.100639 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.100567 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b175d704-58a8-44c5-8e4c-bc83aaba9366-podres\") pod \"perf-node-gather-daemonset-swb7w\" (UID: \"b175d704-58a8-44c5-8e4c-bc83aaba9366\") " pod="openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w" Jul 07 15:45:24.100639 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.100624 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b175d704-58a8-44c5-8e4c-bc83aaba9366-lib-modules\") pod \"perf-node-gather-daemonset-swb7w\" (UID: \"b175d704-58a8-44c5-8e4c-bc83aaba9366\") " pod="openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w" Jul 07 15:45:24.100791 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.100660 2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b175d704-58a8-44c5-8e4c-bc83aaba9366-proc\") pod \"perf-node-gather-daemonset-swb7w\" (UID: \"b175d704-58a8-44c5-8e4c-bc83aaba9366\") " pod="openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w" Jul 07 15:45:24.201892 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.201829 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b175d704-58a8-44c5-8e4c-bc83aaba9366-sys\") pod \"perf-node-gather-daemonset-swb7w\" (UID: \"b175d704-58a8-44c5-8e4c-bc83aaba9366\") " pod="openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w" Jul 07 15:45:24.202315 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.201929 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k86cm\" (UniqueName: \"kubernetes.io/projected/b175d704-58a8-44c5-8e4c-bc83aaba9366-kube-api-access-k86cm\") pod \"perf-node-gather-daemonset-swb7w\" (UID: \"b175d704-58a8-44c5-8e4c-bc83aaba9366\") " pod="openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w" Jul 07 15:45:24.202315 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.201960 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b175d704-58a8-44c5-8e4c-bc83aaba9366-sys\") pod \"perf-node-gather-daemonset-swb7w\" (UID: \"b175d704-58a8-44c5-8e4c-bc83aaba9366\") " pod="openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w" Jul 07 15:45:24.202315 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.201984 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b175d704-58a8-44c5-8e4c-bc83aaba9366-podres\") pod \"perf-node-gather-daemonset-swb7w\" (UID: \"b175d704-58a8-44c5-8e4c-bc83aaba9366\") " pod="openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w" Jul 07 15:45:24.202315 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.202031 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b175d704-58a8-44c5-8e4c-bc83aaba9366-lib-modules\") pod \"perf-node-gather-daemonset-swb7w\" (UID: \"b175d704-58a8-44c5-8e4c-bc83aaba9366\") " pod="openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w" Jul 07 15:45:24.202315 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.202061 2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b175d704-58a8-44c5-8e4c-bc83aaba9366-proc\") pod \"perf-node-gather-daemonset-swb7w\" (UID: \"b175d704-58a8-44c5-8e4c-bc83aaba9366\") " pod="openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w" Jul 07 15:45:24.202315 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.202137 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b175d704-58a8-44c5-8e4c-bc83aaba9366-podres\") pod \"perf-node-gather-daemonset-swb7w\" (UID: \"b175d704-58a8-44c5-8e4c-bc83aaba9366\") " pod="openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w" Jul 07 15:45:24.202315 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.202149 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b175d704-58a8-44c5-8e4c-bc83aaba9366-proc\") pod \"perf-node-gather-daemonset-swb7w\" (UID: \"b175d704-58a8-44c5-8e4c-bc83aaba9366\") " pod="openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w" Jul 07 15:45:24.202315 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.202188 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b175d704-58a8-44c5-8e4c-bc83aaba9366-lib-modules\") pod \"perf-node-gather-daemonset-swb7w\" (UID: \"b175d704-58a8-44c5-8e4c-bc83aaba9366\") " pod="openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w" Jul 07 15:45:24.210102 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.210073 2556 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k86cm\" (UniqueName: \"kubernetes.io/projected/b175d704-58a8-44c5-8e4c-bc83aaba9366-kube-api-access-k86cm\") pod \"perf-node-gather-daemonset-swb7w\" (UID: \"b175d704-58a8-44c5-8e4c-bc83aaba9366\") " pod="openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w" Jul 07 15:45:24.319812 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.319723 2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w" Jul 07 15:45:24.458589 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.458562 2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w"] Jul 07 15:45:24.461197 ip-10-0-130-39 kubenswrapper[2556]: W0707 15:45:24.461168 2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb175d704_58a8_44c5_8e4c_bc83aaba9366.slice/crio-099154e6e7141ee7b06ac6e29641fdf6b43f5889959e62d59046341e72bd0ad6 WatchSource:0}: Error finding container 099154e6e7141ee7b06ac6e29641fdf6b43f5889959e62d59046341e72bd0ad6: Status 404 returned error can't find the container with id 099154e6e7141ee7b06ac6e29641fdf6b43f5889959e62d59046341e72bd0ad6 Jul 07 15:45:24.982753 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.982719 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w" event={"ID":"b175d704-58a8-44c5-8e4c-bc83aaba9366","Type":"ContainerStarted","Data":"c9da2efd4c73db531f97634f0c3c3987fb788b5c16be35d4e4fe98cf644cc5e8"} Jul 07 15:45:24.982968 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.982780 2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w" event={"ID":"b175d704-58a8-44c5-8e4c-bc83aaba9366","Type":"ContainerStarted","Data":"099154e6e7141ee7b06ac6e29641fdf6b43f5889959e62d59046341e72bd0ad6"} Jul 07 15:45:24.982968 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:24.982807 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w" Jul 07 15:45:25.000053 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:25.000006 2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w" podStartSLOduration=1.9999909470000001 podStartE2EDuration="1.999990947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 15:45:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 15:45:24.998103029 +0000 UTC m=+2673.814369070" watchObservedRunningTime="2026-07-07 15:45:24.999990947 +0000 UTC m=+2673.816256985" Jul 07 15:45:25.528664 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:25.528633 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l7998_1266e9ff-4ff3-4177-8402-1550a338121a/dns/0.log" Jul 07 15:45:25.547036 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:25.547006 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l7998_1266e9ff-4ff3-4177-8402-1550a338121a/kube-rbac-proxy/0.log" Jul 07 15:45:25.661168 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:25.661135 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-spm2h_31976396-202a-47d9-b31f-9bf2efdfd835/dns-node-resolver/0.log" Jul 07 15:45:26.177341 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:26.177307 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7884655bfd-48cr4_e4a1788c-8980-4bbc-a37c-387560ce6533/registry/0.log" Jul 07 15:45:26.247828 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:26.247790 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-w6w74_da182a28-bbef-4bfd-9273-b94f43794676/node-ca/0.log" Jul 07 15:45:27.364642 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:27.364610 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-482t6_98339dff-159d-40b8-b7f1-81492771670d/discovery/0.log" Jul 07 15:45:27.580723 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:27.580688 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-qn6gm_117c8485-434b-4a08-9a53-4883af58af7a/payload-processing/0.log" Jul 07 15:45:27.608509 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:27.608480 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c76867bd5-72zmj_0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7/router/0.log" Jul 07 15:45:28.262532 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:28.262501 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cm8wk_b040cb56-952e-4172-a484-89ba96e76a45/serve-healthcheck-canary/0.log" Jul 07 15:45:28.804305 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:28.804234 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-7qqkl_89ce2379-8dcb-4fde-8423-47dc9bf4512f/insights-operator/0.log" Jul 07 15:45:28.807282 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:28.807249 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-7qqkl_89ce2379-8dcb-4fde-8423-47dc9bf4512f/insights-operator/1.log" Jul 07 15:45:28.829335 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:28.829307 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7lnj4_f1529a40-d0ce-481c-8233-200713ff5d42/kube-rbac-proxy/0.log" Jul 07 15:45:28.851062 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:28.851033 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7lnj4_f1529a40-d0ce-481c-8233-200713ff5d42/exporter/0.log" Jul 07 15:45:28.871944 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:28.871917 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7lnj4_f1529a40-d0ce-481c-8233-200713ff5d42/extractor/0.log" Jul 07 15:45:30.996524 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:30.996498 2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-d8xvs/perf-node-gather-daemonset-swb7w" Jul 07 15:45:31.296907 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:31.296797 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723955-8wgxz_3bca2b6e-9022-4c85-a582-e89a7d8f55fc/cleanup/0.log" Jul 07 15:45:31.318241 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:31.318214 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723970-lqpkr_2ae2ae33-f844-4d0e-84aa-8ded7d7b8c49/cleanup/0.log" Jul 07 15:45:31.343831 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:31.343802 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723985-74vdm_e7df1e71-db4a-4023-b1d7-29c0a4905471/cleanup/0.log" Jul 07 15:45:31.704180 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:31.704152 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6_728e97a8-e828-46f0-bb12-6aa37b215e9a/manager/0.log" Jul 07 15:45:31.724778 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:31.724757 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78cfd9ffc9-qsnv6_728e97a8-e828-46f0-bb12-6aa37b215e9a/copy-manifests/0.log" Jul 07 15:45:33.118853 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:33.118816 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-55947775b5-n2mhz_fdc083d3-a349-401f-ab70-6157042df018/manager/0.log" Jul 07 15:45:34.271744 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:34.271710 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-482t6_98339dff-159d-40b8-b7f1-81492771670d/discovery/0.log" Jul 07 15:45:34.438223 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:34.438144 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-qn6gm_117c8485-434b-4a08-9a53-4883af58af7a/payload-processing/0.log" Jul 07 15:45:34.451912 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:34.451888 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c76867bd5-72zmj_0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7/router/0.log" Jul 07 15:45:35.956513 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:35.956476 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-482t6_98339dff-159d-40b8-b7f1-81492771670d/discovery/0.log" Jul 07 15:45:36.124188 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:36.124151 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-qn6gm_117c8485-434b-4a08-9a53-4883af58af7a/payload-processing/0.log" Jul 07 15:45:36.148519 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:36.148479 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5c76867bd5-72zmj_0952af4f-f3dd-4bdf-b0da-5c0d66fb57b7/router/0.log" Jul 07 15:45:38.934621 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:38.934582 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-wc9cn_21e316ac-2667-4c4e-b726-88a3c866eb77/kube-storage-version-migrator-operator/1.log" Jul 07 15:45:38.936065 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:38.936038 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-wc9cn_21e316ac-2667-4c4e-b726-88a3c866eb77/kube-storage-version-migrator-operator/0.log" Jul 07 15:45:40.062700 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:40.062669 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lzp4x_0a6a7a17-0740-428b-b94c-e073a570c9a7/kube-multus-additional-cni-plugins/0.log" Jul 07 15:45:40.081909 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:40.081850 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lzp4x_0a6a7a17-0740-428b-b94c-e073a570c9a7/egress-router-binary-copy/0.log" Jul 07 15:45:40.098951 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:40.098922 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lzp4x_0a6a7a17-0740-428b-b94c-e073a570c9a7/cni-plugins/0.log" Jul 07 15:45:40.117934 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:40.117911 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lzp4x_0a6a7a17-0740-428b-b94c-e073a570c9a7/bond-cni-plugin/0.log" Jul 07 15:45:40.138178 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:40.138155 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lzp4x_0a6a7a17-0740-428b-b94c-e073a570c9a7/routeoverride-cni/0.log" Jul 07 15:45:40.156130 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:40.156096 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lzp4x_0a6a7a17-0740-428b-b94c-e073a570c9a7/whereabouts-cni-bincopy/0.log" Jul 07 15:45:40.176730 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:40.176709 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lzp4x_0a6a7a17-0740-428b-b94c-e073a570c9a7/whereabouts-cni/0.log" Jul 07 15:45:40.404015 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:40.403983 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vxlhb_8466910e-c944-4c88-914b-596d470f31a6/kube-multus/0.log" Jul 07 15:45:40.505345 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:40.505314 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dq8k2_f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a/network-metrics-daemon/0.log" Jul 07 15:45:40.527796 ip-10-0-130-39 kubenswrapper[2556]: I0707 15:45:40.527765 2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dq8k2_f6f81ab4-1ef1-4fdf-b6c7-50023a27d54a/kube-rbac-proxy/0.log"