Nov 04 05:26:31.407312 ip-10-0-130-185 systemd[1]: Starting Kubernetes Kubelet... Nov 04 05:26:31.933149 ip-10-0-130-185 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:26:31.933149 ip-10-0-130-185 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Nov 04 05:26:31.933149 ip-10-0-130-185 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:26:31.933149 ip-10-0-130-185 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:26:31.945638 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945534 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:31.945638 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945635 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:31.945638 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945639 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:31.945638 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945642 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:31.945751 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945645 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:31.945751 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945648 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:31.945751 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945651 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:31.945751 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945654 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:31.945751 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945657 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:31.945751 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945659 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:31.945751 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945662 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:31.945751 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945664 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:31.945751 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945668 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:31.945751 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945670 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:31.945751 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945673 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:31.945751 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945676 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:31.945751 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945678 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:31.945751 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945682 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:31.945751 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945685 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:31.945751 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945688 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:31.945751 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945690 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:31.945751 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945693 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:31.945751 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945696 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:31.945751 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945699 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:31.946234 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945701 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:31.946234 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945704 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:31.946234 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945706 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:31.946234 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945711 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:31.946234 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945715 2575 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:31.946234 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945718 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:31.946234 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945721 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:31.946234 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945723 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:31.946234 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945726 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:31.946234 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945729 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:31.946234 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945731 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:31.946234 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945734 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:31.946234 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945737 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:31.946234 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945740 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:31.946234 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945743 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:31.946234 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945746 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:31.946234 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945749 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:31.946234 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945753 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:31.946234 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945757 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:31.946681 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945760 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:31.946681 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945763 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:31.946681 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945766 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:31.946681 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945769 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:31.946681 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945772 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:31.946681 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945775 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:31.946681 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945777 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:31.946681 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945780 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:31.946681 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945782 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:31.946681 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945785 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:31.946681 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945787 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:31.946681 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945790 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:31.946681 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945793 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:31.946681 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945795 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:31.946681 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945798 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:31.946681 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945801 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:31.946681 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945803 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:31.946681 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945807 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:31.946681 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945809 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:31.946681 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945812 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:31.947163 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945815 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:31.947163 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945817 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:31.947163 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945820 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:31.947163 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945823 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:31.947163 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945826 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:31.947163 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945828 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:31.947163 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945831 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:31.947163 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945834 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:31.947163 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945837 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:31.947163 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945841 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:31.947163 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945843 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:31.947163 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945846 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:31.947163 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945848 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:31.947163 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945851 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:31.947163 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945854 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:31.947163 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945856 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:31.947163 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945859 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:31.947163 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945861 2575 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:31.947163 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945864 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:31.947640 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945866 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:31.947640 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945869 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:31.947640 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945871 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:31.947640 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945874 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:31.947640 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945878 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:31.947640 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945881 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:31.947640 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945883 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:31.947640 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945886 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:31.947640 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945889 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:31.947640 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945891 2575 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:31.947640 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945894 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:31.947640 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945897 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:31.947640 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945900 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:31.947640 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945902 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:31.947640 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945905 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:31.947640 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945907 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:31.947640 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945910 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:31.947640 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945912 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:31.947640 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945915 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:31.947640 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945918 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:31.948152 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945920 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:31.948152 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945923 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:31.948152 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945926 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:31.948152 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.945930 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:31.948152 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946768 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:31.948152 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946776 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:31.948152 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946779 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:31.948152 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946782 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:31.948152 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946784 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:31.948152 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946787 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:31.948152 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946790 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:31.948152 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946793 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:31.948152 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946796 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:31.948152 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946799 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:31.948152 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946802 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:31.948152 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946805 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:31.948152 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946808 2575 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:31.948152 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946810 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:31.948152 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946813 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:31.948152 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946816 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:31.948664 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946819 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:31.948664 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946821 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:31.948664 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946824 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:31.948664 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946827 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:31.948664 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946830 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:31.948664 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946834 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:31.948664 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946837 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:31.948664 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946840 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:31.948664 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946843 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:31.948664 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946846 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:31.948664 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946850 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:31.948664 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946852 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:31.948664 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946855 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:31.948664 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946857 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:31.948664 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946860 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:31.948664 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946863 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:31.948664 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946871 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:31.948664 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946874 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:31.948664 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946877 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:31.949129 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946880 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:31.949129 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946882 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:31.949129 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946885 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:31.949129 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946888 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:31.949129 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946890 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:31.949129 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946893 2575 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:31.949129 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946896 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:31.949129 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946898 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:31.949129 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946901 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:31.949129 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946903 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:31.949129 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946906 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:31.949129 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946910 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:31.949129 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946914 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:31.949129 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946917 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:31.949129 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946920 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:31.949129 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946922 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:31.949129 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946925 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:31.949129 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946927 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:31.949129 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946930 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:31.949685 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946932 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:31.949685 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946935 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:31.949685 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946938 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:31.949685 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946940 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:31.949685 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946943 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:31.949685 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946945 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:31.949685 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946948 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:31.949685 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946951 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:31.949685 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946953 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:31.949685 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946956 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:31.949685 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946958 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:31.949685 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946976 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:31.949685 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946979 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:31.949685 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946981 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:31.949685 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946984 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:31.949685 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946987 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:31.949685 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946990 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:31.949685 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946992 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:31.949685 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946995 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:31.950196 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.946997 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:31.950196 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947000 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:31.950196 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947003 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:31.950196 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947005 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:31.950196 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947008 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:31.950196 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947011 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:31.950196 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947013 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:31.950196 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947016 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:31.950196 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947018 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:31.950196 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947021 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:31.950196 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947023 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:31.950196 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947027 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:31.950196 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947029 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:31.950196 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947052 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:31.950196 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947055 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:31.950196 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947057 2575 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:31.950196 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947061 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:31.950196 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947063 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:31.950196 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947066 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:31.950196 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947069 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:31.950677 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947071 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:31.950677 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947074 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:31.950677 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947081 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:31.950677 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947083 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:31.950677 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947086 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:31.950677 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947089 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:31.950677 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947092 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:31.950677 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947094 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:31.950677 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947097 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:31.950677 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947102 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:31.950677 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947104 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:31.950677 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947107 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:31.950677 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947109 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:31.950677 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947638 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:31.950677 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947644 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:31.950677 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947647 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:31.950677 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947650 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:31.950677 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947653 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:31.950677 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947656 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:31.951139 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947658 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:31.951139 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947661 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:31.951139 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947664 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:31.951139 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947667 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:31.951139 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947670 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:31.951139 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947673 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:31.951139 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947675 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:31.951139 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947678 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:31.951139 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947681 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:31.951139 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947684 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:31.951139 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947686 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:31.951139 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947689 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:31.951139 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947692 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:31.951139 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947694 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:31.951139 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947697 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:31.951139 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947699 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:31.951139 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947702 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:31.951139 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947704 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:31.951139 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947708 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:31.951139 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947711 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:31.951619 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947713 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:31.951619 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947716 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:31.951619 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947719 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:31.951619 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947722 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:31.951619 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947725 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:31.951619 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947727 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:31.951619 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947730 2575 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:31.951619 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947732 2575 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:31.951619 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947735 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:31.951619 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947737 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:31.951619 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947740 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:31.951619 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947742 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:31.951619 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947745 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:31.951619 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947747 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:31.951619 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947750 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:31.951619 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947753 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:31.951619 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947755 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:31.951619 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947757 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:31.951619 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947760 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:31.951619 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947763 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:31.952102 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947765 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:31.952102 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947768 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:31.952102 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947770 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:31.952102 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947773 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:31.952102 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947775 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:31.952102 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947778 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:31.952102 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947781 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:31.952102 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947783 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:31.952102 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947785 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:31.952102 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947788 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:31.952102 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947790 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:31.952102 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947794 2575 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:31.952102 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947797 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:31.952102 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947800 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:31.952102 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947804 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:31.952102 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947807 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:31.952102 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947810 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:31.952102 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947812 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:31.952102 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947815 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:31.952540 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947818 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:31.952540 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947820 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:31.952540 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947823 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:31.952540 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947826 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:31.952540 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947828 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:31.952540 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947831 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:31.952540 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947833 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:31.952540 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947836 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:31.952540 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947838 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:31.952540 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947841 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:31.952540 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947844 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:31.952540 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947846 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:31.952540 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947849 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:31.952540 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947852 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:31.952540 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947855 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:31.952540 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947859 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:31.952540 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947862 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:31.952540 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947864 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:31.952540 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947867 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:31.952540 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947870 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:31.953013 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947872 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:31.953013 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947874 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:31.953013 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947877 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:31.953013 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947879 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:31.953013 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947882 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:31.953013 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947884 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:31.953013 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947887 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:31.953013 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947889 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:31.953013 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947892 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:31.953013 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947895 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:31.953013 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947899 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:31.953013 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947902 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:31.953013 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947904 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:31.953013 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947907 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:31.953013 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947910 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:31.953013 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947912 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:31.953013 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947914 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:31.953013 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947917 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:31.953013 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947919 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947922 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.947924 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948849 2575 flags.go:64] FLAG: --address="0.0.0.0" Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948858 2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948866 2575 flags.go:64] FLAG: --anonymous-auth="true" Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948871 2575 flags.go:64] FLAG: --application-metrics-count-limit="100" Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948876 2575 flags.go:64] FLAG: --authentication-token-webhook="false" Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948879 2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948886 2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948890 2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948894 2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948897 2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948901 2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948905 2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948908 2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948911 2575 flags.go:64] FLAG: --cgroup-root="" Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948914 2575 flags.go:64] FLAG: --cgroups-per-qos="true" Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948917 2575 flags.go:64] FLAG: --client-ca-file="" Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948920 2575 flags.go:64] FLAG: --cloud-provider="external" Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948929 2575 flags.go:64] FLAG: --cluster-dns="[]" Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948934 2575 flags.go:64] FLAG: --cluster-domain="" Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948937 2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Nov 04 05:26:31.953465 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948940 2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948943 2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948946 2575 flags.go:64] FLAG: --container-log-max-files="5" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948950 2575 flags.go:64] FLAG: --container-log-max-size="10Mi" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948953 2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948956 2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948960 2575 flags.go:64] FLAG: --containerd-namespace="k8s.io" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948962 2575 flags.go:64] FLAG: --contention-profiling="false" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948965 2575 flags.go:64] FLAG: --cpu-cfs-quota="true" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948969 2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948972 2575 flags.go:64] FLAG: --cpu-manager-policy="none" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948975 2575 flags.go:64] FLAG: --cpu-manager-policy-options="" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948979 2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948982 2575 flags.go:64] FLAG: --enable-controller-attach-detach="true" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948985 2575 flags.go:64] FLAG: --enable-debugging-handlers="true" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948990 2575 flags.go:64] FLAG: --enable-load-reader="false" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948993 2575 flags.go:64] FLAG: --enable-server="true" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.948996 2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949005 2575 flags.go:64] FLAG: --event-burst="100" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949008 2575 flags.go:64] FLAG: --event-qps="50" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949011 2575 flags.go:64] FLAG: --event-storage-age-limit="default=0" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949014 2575 flags.go:64] FLAG: --event-storage-event-limit="default=0" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949017 2575 flags.go:64] FLAG: --eviction-hard="" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949022 2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Nov 04 05:26:31.954006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949025 2575 flags.go:64] FLAG: --eviction-minimum-reclaim="" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949028 2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949043 2575 flags.go:64] FLAG: --eviction-soft="" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949047 2575 flags.go:64] FLAG: --eviction-soft-grace-period="" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949050 2575 flags.go:64] FLAG: --exit-on-lock-contention="false" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949053 2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949062 2575 flags.go:64] FLAG: --experimental-mounter-path="" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949065 2575 flags.go:64] FLAG: --fail-cgroupv1="true" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949068 2575 flags.go:64] FLAG: --fail-swap-on="true" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949071 2575 flags.go:64] FLAG: --feature-gates="" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949075 2575 flags.go:64] FLAG: --file-check-frequency="20s" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949078 2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949081 2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949084 2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949087 2575 flags.go:64] FLAG: --healthz-port="10248" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949091 2575 flags.go:64] FLAG: --help="false" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949094 2575 flags.go:64] FLAG: --hostname-override="ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949097 2575 flags.go:64] FLAG: --housekeeping-interval="10s" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949100 2575 flags.go:64] FLAG: --http-check-frequency="20s" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949103 2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949106 2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949110 2575 flags.go:64] FLAG: --image-gc-high-threshold="85" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949113 2575 flags.go:64] FLAG: --image-gc-low-threshold="80" Nov 04 05:26:31.954686 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949117 2575 flags.go:64] FLAG: --image-service-endpoint="" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949120 2575 flags.go:64] FLAG: --kernel-memcg-notification="false" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949123 2575 flags.go:64] FLAG: --kube-api-burst="100" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949128 2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949131 2575 flags.go:64] FLAG: --kube-api-qps="50" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949135 2575 flags.go:64] FLAG: --kube-reserved="" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949138 2575 flags.go:64] FLAG: --kube-reserved-cgroup="" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949141 2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949144 2575 flags.go:64] FLAG: --kubelet-cgroups="" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949147 2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949150 2575 flags.go:64] FLAG: --lock-file="" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949153 2575 flags.go:64] FLAG: --log-cadvisor-usage="false" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949156 2575 flags.go:64] FLAG: --log-flush-frequency="5s" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949159 2575 flags.go:64] FLAG: --log-json-info-buffer-size="0" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949164 2575 flags.go:64] FLAG: --log-json-split-stream="false" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949167 2575 flags.go:64] FLAG: --log-text-info-buffer-size="0" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949172 2575 flags.go:64] FLAG: --log-text-split-stream="false" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949175 2575 flags.go:64] FLAG: --logging-format="text" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949178 2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949181 2575 flags.go:64] FLAG: --make-iptables-util-chains="true" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949184 2575 flags.go:64] FLAG: --manifest-url="" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949187 2575 flags.go:64] FLAG: --manifest-url-header="" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949192 2575 flags.go:64] FLAG: --max-housekeeping-interval="15s" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949194 2575 flags.go:64] FLAG: --max-open-files="1000000" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949198 2575 flags.go:64] FLAG: --max-pods="110" Nov 04 05:26:31.955280 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949201 2575 flags.go:64] FLAG: --maximum-dead-containers="-1" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949204 2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949207 2575 flags.go:64] FLAG: --memory-manager-policy="None" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949210 2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949213 2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949216 2575 flags.go:64] FLAG: --node-ip="0.0.0.0" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949219 2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949229 2575 flags.go:64] FLAG: --node-status-max-images="50" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949232 2575 flags.go:64] FLAG: --node-status-update-frequency="10s" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949235 2575 flags.go:64] FLAG: --oom-score-adj="-999" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949239 2575 flags.go:64] FLAG: --pod-cidr="" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949242 2575 flags.go:64] FLAG: --pod-manifest-path="" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949245 2575 flags.go:64] FLAG: --pod-max-pids="-1" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949248 2575 flags.go:64] FLAG: --pods-per-core="0" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949251 2575 flags.go:64] FLAG: --port="10250" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949254 2575 flags.go:64] FLAG: --protect-kernel-defaults="false" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949257 2575 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-0da4f6eccf68ffd9d" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949260 2575 flags.go:64] FLAG: --qos-reserved="" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949263 2575 flags.go:64] FLAG: --read-only-port="10255" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949266 2575 flags.go:64] FLAG: --register-node="true" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949269 2575 flags.go:64] FLAG: --register-with-taints="" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949273 2575 flags.go:64] FLAG: --registry-burst="10" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949276 2575 flags.go:64] FLAG: --registry-qps="5" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949279 2575 flags.go:64] FLAG: --reserved-cpus="" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949282 2575 flags.go:64] FLAG: --reserved-memory="" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949286 2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Nov 04 05:26:31.955872 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949289 2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949292 2575 flags.go:64] FLAG: --rotate-certificates="false" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949295 2575 flags.go:64] FLAG: --rotate-server-certificates="false" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949298 2575 flags.go:64] FLAG: --runonce="false" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949301 2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949304 2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949307 2575 flags.go:64] FLAG: --seccomp-default="false" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949310 2575 flags.go:64] FLAG: --serialize-image-pulls="true" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949313 2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949316 2575 flags.go:64] FLAG: --storage-driver-db="cadvisor" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949319 2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949322 2575 flags.go:64] FLAG: --storage-driver-password="root" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949325 2575 flags.go:64] FLAG: --storage-driver-secure="false" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949329 2575 flags.go:64] FLAG: --storage-driver-table="stats" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949332 2575 flags.go:64] FLAG: --storage-driver-user="root" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949335 2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949339 2575 flags.go:64] FLAG: --sync-frequency="1m0s" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949342 2575 flags.go:64] FLAG: --system-cgroups="" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949345 2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949351 2575 flags.go:64] FLAG: --system-reserved-cgroup="" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949354 2575 flags.go:64] FLAG: --tls-cert-file="" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949357 2575 flags.go:64] FLAG: --tls-cipher-suites="[]" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949364 2575 flags.go:64] FLAG: --tls-min-version="" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949367 2575 flags.go:64] FLAG: --tls-private-key-file="" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949370 2575 flags.go:64] FLAG: --topology-manager-policy="none" Nov 04 05:26:31.956523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949373 2575 flags.go:64] FLAG: --topology-manager-policy-options="" Nov 04 05:26:31.957126 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949376 2575 flags.go:64] FLAG: --topology-manager-scope="container" Nov 04 05:26:31.957126 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949378 2575 flags.go:64] FLAG: --v="2" Nov 04 05:26:31.957126 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949383 2575 flags.go:64] FLAG: --version="false" Nov 04 05:26:31.957126 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949386 2575 flags.go:64] FLAG: --vmodule="" Nov 04 05:26:31.957126 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949390 2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Nov 04 05:26:31.957126 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949397 2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Nov 04 05:26:31.957126 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949538 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:31.957126 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949541 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:31.957126 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949544 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:31.957126 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949547 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:31.957126 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949549 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:31.957126 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949552 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:31.957126 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949555 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:31.957126 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949557 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:31.957126 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949560 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:31.957126 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949563 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:31.957126 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949581 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:31.957126 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949586 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:31.957126 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949589 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:31.957126 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949595 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:31.957734 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949600 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:31.957734 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949604 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:31.957734 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949609 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:31.957734 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949611 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:31.957734 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949614 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:31.957734 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949617 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:31.957734 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949620 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:31.957734 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949623 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:31.957734 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949626 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:31.957734 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949629 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:31.957734 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949631 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:31.957734 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949634 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:31.957734 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949637 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:31.957734 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949639 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:31.957734 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949642 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:31.957734 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949645 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:31.957734 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949647 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:31.957734 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949650 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:31.957734 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949658 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:31.958495 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949661 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:31.958495 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949663 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:31.958495 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949667 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:31.958495 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949671 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:31.958495 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949674 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:31.958495 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949677 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:31.958495 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949679 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:31.958495 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949682 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:31.958495 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949684 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:31.958495 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949687 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:31.958495 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949690 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:31.958495 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949692 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:31.958495 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949696 2575 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:31.958495 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949699 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:31.958495 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949702 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:31.958495 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949705 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:31.958495 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949707 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:31.958495 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949710 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:31.958495 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949712 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:31.958495 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949715 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:31.959017 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949717 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:31.959017 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949720 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:31.959017 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949722 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:31.959017 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949725 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:31.959017 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949727 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:31.959017 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949730 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:31.959017 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949733 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:31.959017 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949735 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:31.959017 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949737 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:31.959017 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949740 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:31.959017 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949742 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:31.959017 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949745 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:31.959017 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949753 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:31.959017 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949756 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:31.959017 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949758 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:31.959017 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949761 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:31.959017 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949763 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:31.959017 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949766 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:31.959017 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949768 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:31.959017 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949771 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:31.959562 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949773 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:31.959562 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949776 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:31.959562 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949778 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:31.959562 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949781 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:31.959562 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949784 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:31.959562 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949787 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:31.959562 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949789 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:31.959562 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949792 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:31.959562 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949795 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:31.959562 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949798 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:31.959562 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949800 2575 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:31.959562 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949803 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:31.959562 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949805 2575 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:31.959562 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949808 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:31.959562 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949811 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:31.959562 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949813 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:31.959562 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949816 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:31.959562 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949819 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:31.959562 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949821 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:31.959562 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949824 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:31.960046 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949826 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:31.960046 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949828 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:31.960046 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949831 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:31.960046 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949834 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:31.960046 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949836 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:31.960046 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949840 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:31.960046 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949843 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:31.960046 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949845 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:31.960046 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949848 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:31.960046 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949850 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:31.960046 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949853 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:31.960046 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949856 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:31.960046 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949858 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:31.960046 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.949884 2575 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:26:31.960046 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949925 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:31.960046 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949930 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:31.960430 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949933 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:31.960430 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949939 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:31.960430 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949941 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:31.960430 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949951 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:31.960430 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949955 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:31.960430 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949958 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:31.960430 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949961 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:31.960430 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949964 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:31.960430 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949966 2575 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:31.960430 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949969 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:31.960430 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949972 2575 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:31.960430 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949974 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:31.960430 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949977 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:31.960430 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949980 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:31.960430 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949982 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:31.960430 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949985 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:31.960430 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949987 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:31.960430 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949990 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:31.960430 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949993 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:31.960430 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949995 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:31.960890 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.949998 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:31.960890 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950000 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:31.960890 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950004 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:31.960890 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950006 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:31.960890 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950009 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:31.960890 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950012 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:31.960890 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950015 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:31.960890 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950017 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:31.960890 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950020 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:31.960890 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950023 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:31.960890 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950025 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:31.960890 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950028 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:31.960890 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950042 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:31.960890 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950047 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:31.960890 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950049 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:31.960890 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950052 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:31.960890 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950054 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:31.960890 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950057 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:31.960890 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950060 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:31.960890 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950062 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:31.961381 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950065 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:31.961381 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950068 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:31.961381 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950070 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:31.961381 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950073 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:31.961381 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950077 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:31.961381 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950081 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:31.961381 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950084 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:31.961381 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950088 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:31.961381 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950090 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:31.961381 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950093 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:31.961381 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950096 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:31.961381 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950099 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:31.961381 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950101 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:31.961381 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950105 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:31.961381 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950108 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:31.961381 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950111 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:31.961381 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950114 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:31.961381 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950116 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:31.961798 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950119 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:31.961798 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950122 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:31.961798 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950124 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:31.961798 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950127 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:31.961798 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950129 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:31.961798 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950132 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:31.961798 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950134 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:31.961798 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950139 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:31.961798 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950143 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:31.961798 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950147 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:31.961798 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950150 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:31.961798 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950153 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:31.961798 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950155 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:31.961798 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950158 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:31.961798 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950161 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:31.961798 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950163 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:31.961798 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950166 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:31.961798 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950168 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:31.961798 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950170 2575 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:31.961798 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950173 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:31.962292 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950176 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:31.962292 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950178 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:31.962292 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950180 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:31.962292 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950183 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:31.962292 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950186 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:31.962292 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950188 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:31.962292 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950191 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:31.962292 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950193 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:31.962292 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950196 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:31.962292 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950198 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:31.962292 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950204 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:31.962292 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950207 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:31.962292 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950209 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:31.962292 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950212 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:31.962292 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950214 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:31.962292 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950217 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:31.962292 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950220 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:31.962292 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950222 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:31.962292 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950225 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:31.962725 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950228 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:31.962725 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950231 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:31.962725 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950235 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:31.962725 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950238 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:31.962725 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950240 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:31.962725 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950244 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:31.962725 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.950246 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:31.962725 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.958801 2575 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Nov 04 05:26:31.962725 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.958817 2575 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Nov 04 05:26:31.962725 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.958835 2575 watchdog_linux.go:95] "Systemd watchdog is not enabled" Nov 04 05:26:31.962725 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.958840 2575 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Nov 04 05:26:31.962725 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958901 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:31.962725 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958907 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:31.962725 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958910 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:31.962725 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958914 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:31.962725 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958917 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:31.962725 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958919 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:31.962725 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958922 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:31.962725 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958925 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:31.963194 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958928 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:31.963194 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958930 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:31.963194 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958933 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:31.963194 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958936 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:31.963194 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958938 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:31.963194 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958941 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:31.963194 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958944 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:31.963194 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958947 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:31.963194 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958949 2575 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:31.963194 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958952 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:31.963194 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958954 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:31.963194 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958957 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:31.963194 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958960 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:31.963194 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958962 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:31.963194 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958965 2575 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:31.963194 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958967 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:31.963194 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958971 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:31.963194 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958974 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:31.963194 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958978 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:31.963194 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958982 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:31.963651 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958985 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:31.963651 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958990 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:31.963651 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958994 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:31.963651 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.958998 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:31.963651 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959002 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:31.963651 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959005 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:31.963651 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959008 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:31.963651 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959010 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:31.963651 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959013 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:31.963651 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959016 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:31.963651 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959018 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:31.963651 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959021 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:31.963651 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959024 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:31.963651 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959027 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:31.963651 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959045 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:31.963651 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959049 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:31.963651 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959052 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:31.963651 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959055 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:31.964190 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959059 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:31.964190 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959062 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:31.964190 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959065 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:31.964190 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959067 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:31.964190 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959070 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:31.964190 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959072 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:31.964190 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959075 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:31.964190 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959078 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:31.964190 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959081 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:31.964190 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959084 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:31.964190 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959087 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:31.964190 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959089 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:31.964190 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959092 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:31.964190 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959095 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:31.964190 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959098 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:31.964190 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959101 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:31.964190 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959103 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:31.964190 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959106 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:31.964190 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959109 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:31.964643 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959112 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:31.964643 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959114 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:31.964643 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959117 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:31.964643 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959120 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:31.964643 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959122 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:31.964643 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959125 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:31.964643 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959128 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:31.964643 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959130 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:31.964643 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959133 2575 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:31.964643 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959135 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:31.964643 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959138 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:31.964643 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959141 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:31.964643 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959143 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:31.964643 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959146 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:31.964643 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959149 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:31.964643 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959152 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:31.964643 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959155 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:31.964643 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959157 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:31.964643 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959160 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:31.964643 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959163 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:31.965165 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959165 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:31.965165 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959168 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:31.965165 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959171 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:31.965165 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959173 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:31.965165 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959176 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:31.965165 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959178 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:31.965165 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959182 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:31.965165 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959184 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:31.965165 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959187 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:31.965165 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959189 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:31.965165 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959192 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:31.965165 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959194 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:31.965165 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959197 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:31.965165 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959199 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:31.965165 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959202 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:31.965165 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959204 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:31.965165 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959208 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:31.965165 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959210 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:31.965165 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959213 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:31.965600 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959216 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:31.965600 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959218 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:31.965600 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.959248 2575 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:26:31.965600 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959393 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:31.965600 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959397 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:31.965600 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959400 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:31.965600 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959403 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:31.965600 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959407 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:31.965600 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959410 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:31.965600 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959413 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:31.965600 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959416 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:31.965600 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959418 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:31.965600 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959421 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:31.965600 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959424 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:31.965600 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959427 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:31.965600 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959430 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:31.965983 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959432 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:31.965983 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959435 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:31.965983 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959438 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:31.965983 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959441 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:31.965983 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959444 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:31.965983 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959446 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:31.965983 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959449 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:31.965983 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959451 2575 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:31.965983 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959454 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:31.965983 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959458 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:31.965983 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959463 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:31.965983 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959466 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:31.965983 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959468 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:31.965983 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959472 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:31.965983 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959475 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:31.965983 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959478 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:31.965983 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959480 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:31.965983 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959483 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:31.965983 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959486 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:31.966435 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959488 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:31.966435 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959491 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:31.966435 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959493 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:31.966435 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959496 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:31.966435 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959498 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:31.966435 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959501 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:31.966435 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959506 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:31.966435 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959509 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:31.966435 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959512 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:31.966435 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959515 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:31.966435 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959517 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:31.966435 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959520 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:31.966435 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959523 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:31.966435 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959525 2575 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:31.966435 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959528 2575 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:31.966435 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959530 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:31.966435 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959533 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:31.966435 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959535 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:31.966435 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959538 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:31.966435 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959540 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:31.966901 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959543 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:31.966901 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959546 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:31.966901 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959548 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:31.966901 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959551 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:31.966901 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959554 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:31.966901 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959557 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:31.966901 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959560 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:31.966901 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959563 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:31.966901 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959565 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:31.966901 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959568 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:31.966901 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959571 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:31.966901 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959574 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:31.966901 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959576 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:31.966901 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959579 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:31.966901 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959582 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:31.966901 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959584 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:31.966901 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959587 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:31.966901 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959589 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:31.966901 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959592 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:31.966901 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959595 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:31.967375 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959598 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:31.967375 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959601 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:31.967375 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959603 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:31.967375 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959606 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:31.967375 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959608 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:31.967375 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959611 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:31.967375 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959613 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:31.967375 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959616 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:31.967375 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959618 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:31.967375 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959621 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:31.967375 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959624 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:31.967375 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959626 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:31.967375 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959629 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:31.967375 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959631 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:31.967375 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959633 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:31.967375 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959636 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:31.967375 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959638 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:31.967375 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959641 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:31.967375 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959644 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:31.967375 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959646 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:31.967847 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959649 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:31.967847 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959651 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:31.967847 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959654 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:31.967847 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959657 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:31.967847 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959659 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:31.967847 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959662 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:31.967847 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959664 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:31.967847 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959667 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:31.967847 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959669 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:31.967847 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959672 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:31.967847 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959674 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:31.967847 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959677 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:31.967847 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959680 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:31.967847 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959683 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:31.967847 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.959706 2575 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:26:31.967847 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959737 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:26:31.968347 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959742 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:26:31.968347 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959746 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:26:31.968347 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959748 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:26:31.968347 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959751 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:26:31.968347 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959753 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:26:31.968347 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959759 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:26:31.968347 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959761 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:26:31.968347 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959764 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:26:31.968347 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959767 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:26:31.968347 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959770 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:26:31.968347 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959772 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:26:31.968347 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959775 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:26:31.968347 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959777 2575 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:26:31.968347 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959782 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:26:31.968347 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959786 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:26:31.968347 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959790 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:26:31.968347 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959793 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:26:31.968347 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959797 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:26:31.968347 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959800 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:26:31.968791 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959804 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:26:31.968791 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959807 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:26:31.968791 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959810 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:26:31.968791 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959812 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:26:31.968791 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959815 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:26:31.968791 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959818 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:26:31.968791 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959821 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:26:31.968791 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959824 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:26:31.968791 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959827 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:26:31.968791 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959830 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:26:31.968791 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959833 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:26:31.968791 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959836 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:26:31.968791 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959839 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:26:31.968791 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959842 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:26:31.968791 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959845 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:26:31.968791 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959847 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:26:31.968791 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959850 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:26:31.968791 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959852 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:26:31.968791 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959856 2575 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:26:31.968791 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959859 2575 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:26:31.969296 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959861 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:26:31.969296 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959864 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:26:31.969296 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959866 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:26:31.969296 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959869 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:26:31.969296 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959872 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:26:31.969296 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959874 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:26:31.969296 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959878 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:26:31.969296 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959881 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:26:31.969296 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959883 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:26:31.969296 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959886 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:26:31.969296 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959889 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:26:31.969296 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959892 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:26:31.969296 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959895 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:26:31.969296 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959897 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:26:31.969296 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959900 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:26:31.969296 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959902 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:26:31.969296 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959905 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:26:31.969296 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959907 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:26:31.969296 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959910 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:26:31.969742 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959913 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:26:31.969742 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959916 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:26:31.969742 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959918 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:26:31.969742 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959921 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:26:31.969742 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959924 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:26:31.969742 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959927 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:26:31.969742 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959929 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:26:31.969742 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959932 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:26:31.969742 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959935 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:26:31.969742 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959937 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:26:31.969742 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959940 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:26:31.969742 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959944 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:26:31.969742 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959946 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:26:31.969742 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959949 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:26:31.969742 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959951 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:26:31.969742 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959954 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:26:31.969742 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959957 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:26:31.969742 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959960 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:26:31.969742 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959962 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:26:31.969742 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959966 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:26:31.970245 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959969 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:26:31.970245 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959972 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:26:31.970245 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959974 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:26:31.970245 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959977 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:26:31.970245 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959980 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:26:31.970245 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959982 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:26:31.970245 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959985 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:26:31.970245 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959988 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:26:31.970245 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959990 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:26:31.970245 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959993 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:26:31.970245 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959995 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:26:31.970245 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.959999 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:26:31.970245 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.960001 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:26:31.970245 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.960004 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:26:31.970245 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.960006 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:26:31.970245 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.960009 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:26:31.970245 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.960011 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:26:31.970245 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.960015 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:26:31.970245 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.960018 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:26:31.970688 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.960020 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:26:31.970688 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.960023 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:26:31.970688 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.960026 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:26:31.970688 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.960028 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:26:31.970688 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.960046 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:26:31.970688 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.960049 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:26:31.970688 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.960052 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:26:31.970688 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:31.960055 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:26:31.970688 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.960113 2575 server.go:957] "Client rotation is on, will bootstrap in background" Nov 04 05:26:31.970688 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.963094 2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Nov 04 05:26:31.970688 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.964100 2575 server.go:1014] "Starting client certificate rotation" Nov 04 05:26:31.970688 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.964195 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 05:26:31.970688 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.964877 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 05:26:31.970688 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.966121 2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 05:26:31.971766 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.971752 2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 05:26:31.983008 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.982991 2575 log.go:25] "Validated CRI v1 runtime API" Nov 04 05:26:31.986962 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.986949 2575 log.go:25] "Validated CRI v1 image API" Nov 04 05:26:31.987778 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.987754 2575 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Nov 04 05:26:31.992168 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.992148 2575 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a3e82bfd-4010-41f9-be4c-a0e2c0261e88:/dev/nvme0n1p4 ff3690df-fbfb-4f34-b429-96733430703f:/dev/nvme0n1p3] Nov 04 05:26:31.992168 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.992167 2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Nov 04 05:26:31.997818 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.997697 2575 manager.go:217] Machine: {Timestamp:2026-11-04 05:26:31.99624228 +0000 UTC m=+0.440891978 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100218 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2037d248ddaebca97eee23effbd6de SystemUUID:ec2037d2-48dd-aebc-a97e-ee23effbd6de BootID:65c2727b-fa96-4e73-820a-4fd3a022ab2c Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:fe:41:f2:0b Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:fe:41:f2:0b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b2:2b:a1:a0:71:eb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Nov 04 05:26:31.998663 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.998651 2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Nov 04 05:26:31.998785 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:31.998769 2575 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Nov 04 05:26:32.002609 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.002584 2575 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Nov 04 05:26:32.002741 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.002611 2575 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-185.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Nov 04 05:26:32.005652 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.005642 2575 topology_manager.go:143] "Creating topology manager with none policy" Nov 04 05:26:32.005684 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.005654 2575 container_manager_linux.go:311] "Creating device plugin manager" Nov 04 05:26:32.005684 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.005669 2575 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 05:26:32.005684 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.005680 2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 05:26:32.006794 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.006784 2575 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Nov 04 05:26:32.011669 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.011658 2575 state_mem.go:41] "Initialized" logger="CPUManager state memory" Nov 04 05:26:32.011778 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.011769 2575 server.go:1235] "Using root directory" path="/var/lib/kubelet" Nov 04 05:26:32.012402 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.012393 2575 kubelet.go:493] "Attempting to sync node with API server" Nov 04 05:26:32.012432 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.012408 2575 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Nov 04 05:26:32.012432 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.012421 2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Nov 04 05:26:32.012432 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.012432 2575 kubelet.go:405] "Adding apiserver pod source" Nov 04 05:26:32.012536 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.012440 2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Nov 04 05:26:32.012902 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.012885 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 05:26:32.018308 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.018290 2575 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Nov 04 05:26:32.023659 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.023646 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 05:26:32.025700 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.025687 2575 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Nov 04 05:26:32.025747 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.025718 2575 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Nov 04 05:26:32.025882 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.025872 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Nov 04 05:26:32.025919 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.025888 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Nov 04 05:26:32.025919 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.025895 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Nov 04 05:26:32.025919 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.025900 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Nov 04 05:26:32.025919 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.025906 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Nov 04 05:26:32.025919 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.025911 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Nov 04 05:26:32.027290 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.027280 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Nov 04 05:26:32.027290 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.027290 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Nov 04 05:26:32.027353 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.027298 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Nov 04 05:26:32.027353 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.027304 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Nov 04 05:26:32.027353 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.027313 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Nov 04 05:26:32.027353 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.027322 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Nov 04 05:26:32.028172 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.028161 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Nov 04 05:26:32.028172 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.028171 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Nov 04 05:26:32.031935 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.031922 2575 server.go:1263] "Started kubelet" Nov 04 05:26:32.032073 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.032011 2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Nov 04 05:26:32.032114 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.032097 2575 server_v1.go:49] "podresources" method="list" useActivePods=true Nov 04 05:26:32.032164 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.032022 2575 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Nov 04 05:26:32.032730 ip-10-0-130-185 systemd[1]: Started Kubernetes Kubelet. Nov 04 05:26:32.033407 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.033368 2575 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Nov 04 05:26:32.034723 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.034707 2575 server.go:317] "Adding debug handlers to kubelet server" Nov 04 05:26:32.038770 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:32.038743 2575 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Nov 04 05:26:32.040022 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.040006 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Nov 04 05:26:32.041371 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.041357 2575 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Nov 04 05:26:32.041468 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:32.041443 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-185.us-west-2.compute.internal\" not found" Nov 04 05:26:32.042070 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.042048 2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Nov 04 05:26:32.042070 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.042053 2575 volume_manager.go:309] "The desired_state_of_world populator starts" Nov 04 05:26:32.042196 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.042079 2575 volume_manager.go:311] "Starting Kubelet Volume Manager" Nov 04 05:26:32.042275 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.042257 2575 reconstruct.go:97] "Volume reconstruction finished" Nov 04 05:26:32.042384 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.042272 2575 reconciler.go:29] "Reconciler: start to sync state" Nov 04 05:26:32.042384 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.042335 2575 factory.go:55] Registering systemd factory Nov 04 05:26:32.042384 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.042348 2575 factory.go:223] Registration of the systemd container factory successfully Nov 04 05:26:32.042547 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.042534 2575 factory.go:153] Registering CRI-O factory Nov 04 05:26:32.042547 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.042547 2575 factory.go:223] Registration of the crio container factory successfully Nov 04 05:26:32.042643 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.042610 2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Nov 04 05:26:32.042643 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.042631 2575 factory.go:103] Registering Raw factory Nov 04 05:26:32.042643 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.042640 2575 manager.go:1199] Started watching for new ooms in manager Nov 04 05:26:32.043683 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.043669 2575 manager.go:319] Starting recovery of all containers Nov 04 05:26:32.057487 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.057473 2575 manager.go:324] Recovery completed Nov 04 05:26:32.061150 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.061138 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:32.062982 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.062958 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:32.063071 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.063000 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:32.063071 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.063015 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:32.063543 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.063527 2575 cpu_manager.go:226] "Starting" policy="none" Nov 04 05:26:32.063543 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.063541 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:32.063669 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.063541 2575 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Nov 04 05:26:32.063669 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.063615 2575 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Nov 04 05:26:32.064189 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.064172 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:32.064255 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.064198 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:32.064255 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.064208 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:32.065868 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.065855 2575 policy_none.go:50] "Start" Nov 04 05:26:32.065930 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.065874 2575 memory_manager.go:187] "Starting memorymanager" policy="None" Nov 04 05:26:32.065930 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.065892 2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Nov 04 05:26:32.068064 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.068053 2575 policy_none.go:44] "Start" Nov 04 05:26:32.125016 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.107625 2575 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Nov 04 05:26:32.125016 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.111821 2575 manager.go:341] "Starting Device Plugin manager" Nov 04 05:26:32.125016 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:32.112606 2575 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Nov 04 05:26:32.125016 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.112618 2575 server.go:87] "Starting device plugin registration server" Nov 04 05:26:32.125016 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.112853 2575 eviction_manager.go:194] "Eviction manager: starting control loop" Nov 04 05:26:32.125016 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.112863 2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Nov 04 05:26:32.125016 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.112943 2575 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Nov 04 05:26:32.125016 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.113052 2575 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Nov 04 05:26:32.125016 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.113062 2575 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Nov 04 05:26:32.125016 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:32.113605 2575 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Nov 04 05:26:32.125016 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:32.113643 2575 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-185.us-west-2.compute.internal\" not found" Nov 04 05:26:32.125016 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.121054 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Nov 04 05:26:32.125016 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.122311 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Nov 04 05:26:32.125016 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.122951 2575 status_manager.go:249] "Starting to sync pod status with apiserver" Nov 04 05:26:32.125016 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.122999 2575 kubelet.go:2521] "Starting kubelet main sync loop" Nov 04 05:26:32.125016 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:32.123059 2575 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Nov 04 05:26:32.167621 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.167554 2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-185.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 04 05:26:32.167621 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:32.167564 2575 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-185.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Nov 04 05:26:32.168819 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:32.167551 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-185.us-west-2.compute.internal.18e4c04dcfd10016 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-185.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-185.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 05:26:32.031895574 +0000 UTC m=+0.476545271,LastTimestamp:2026-11-04 05:26:32.031895574 +0000 UTC m=+0.476545271,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-185.us-west-2.compute.internal,}" Nov 04 05:26:32.213650 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.213628 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:32.214549 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.214533 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:32.214618 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.214557 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:32.214618 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.214567 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:32.214618 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.214602 2575 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:26:32.223803 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.223780 2575 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal"] Nov 04 05:26:32.223868 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.223843 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:32.224645 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.224630 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:32.224721 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.224656 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:32.224721 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.224673 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:32.226519 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.226508 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:32.226665 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.226649 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:26:32.226754 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.226688 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:32.227307 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.227285 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:32.227388 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.227326 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:32.227388 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.227349 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:32.227388 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.227358 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:32.227388 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.227375 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:32.227388 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.227390 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:32.229100 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.229084 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:26:32.229167 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.229115 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:32.229869 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.229852 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:32.229930 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.229882 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:32.229930 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.229895 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:32.231126 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.231110 2575 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Nov 04 05:26:32.231192 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.231159 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nsltp" Nov 04 05:26:32.238345 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:32.238321 2575 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-185.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" Nov 04 05:26:32.238345 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:32.238272 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-185.us-west-2.compute.internal.18e4c04dd1ab59f0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-185.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-185.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-185.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 05:26:32.06298264 +0000 UTC m=+0.507632341,LastTimestamp:2026-11-04 05:26:32.06298264 +0000 UTC m=+0.507632341,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-185.us-west-2.compute.internal,}" Nov 04 05:26:32.238698 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:32.238673 2575 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" Nov 04 05:26:32.243295 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.243277 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dbfe51fb7c8cecce5bf77e4c5e14a1c8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal\" (UID: \"dbfe51fb7c8cecce5bf77e4c5e14a1c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:26:32.243364 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.243303 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dbfe51fb7c8cecce5bf77e4c5e14a1c8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal\" (UID: \"dbfe51fb7c8cecce5bf77e4c5e14a1c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:26:32.243364 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.243319 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4eff7615010b2db89d82737c9bc2bd5b-config\") pod \"kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal\" (UID: \"4eff7615010b2db89d82737c9bc2bd5b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:26:32.254049 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:32.254021 2575 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-185.us-west-2.compute.internal\" not found" node="ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:26:32.256877 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:32.256853 2575 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-185.us-west-2.compute.internal\" not found" node="ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:26:32.292752 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:32.292725 2575 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" Nov 04 05:26:32.314777 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.314754 2575 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:26:32.323441 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.323421 2575 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:26:32.323493 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:32.323446 2575 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-185.us-west-2.compute.internal\": node \"ip-10-0-130-185.us-west-2.compute.internal\" not found" Nov 04 05:26:32.344327 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.344303 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dbfe51fb7c8cecce5bf77e4c5e14a1c8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal\" (UID: \"dbfe51fb7c8cecce5bf77e4c5e14a1c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:26:32.344407 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.344329 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4eff7615010b2db89d82737c9bc2bd5b-config\") pod \"kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal\" (UID: \"4eff7615010b2db89d82737c9bc2bd5b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:26:32.344407 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.344354 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dbfe51fb7c8cecce5bf77e4c5e14a1c8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal\" (UID: \"dbfe51fb7c8cecce5bf77e4c5e14a1c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:26:32.344407 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.344379 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dbfe51fb7c8cecce5bf77e4c5e14a1c8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal\" (UID: \"dbfe51fb7c8cecce5bf77e4c5e14a1c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:26:32.344407 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.344394 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dbfe51fb7c8cecce5bf77e4c5e14a1c8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal\" (UID: \"dbfe51fb7c8cecce5bf77e4c5e14a1c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:26:32.344524 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.344396 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4eff7615010b2db89d82737c9bc2bd5b-config\") pod \"kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal\" (UID: \"4eff7615010b2db89d82737c9bc2bd5b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:26:32.455028 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:32.454945 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-185.us-west-2.compute.internal\" not found" Nov 04 05:26:32.556022 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:32.555987 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-185.us-west-2.compute.internal\" not found" Nov 04 05:26:32.558269 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.558246 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:26:32.560618 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.560598 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:26:32.656572 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:32.656530 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-185.us-west-2.compute.internal\" not found" Nov 04 05:26:32.757469 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:32.757386 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-185.us-west-2.compute.internal\" not found" Nov 04 05:26:32.857950 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:32.857912 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-185.us-west-2.compute.internal\" not found" Nov 04 05:26:32.958189 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:32.958153 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-185.us-west-2.compute.internal\" not found" Nov 04 05:26:32.963695 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.963667 2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Nov 04 05:26:32.963871 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:32.963848 2575 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Nov 04 05:26:32.964009 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:32.963911 2575 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a6110e2483d664bf1a253f79dfb04657-a868372cdfab352f.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-130-185.us-west-2.compute.internal.18e4c04dd1abb78a\": read tcp 10.0.130.185:55300->18.188.242.248:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-130-185.us-west-2.compute.internal.18e4c04dd1abb78a default 5172 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-185.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-185.us-west-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-185.us-west-2.compute.internal,},FirstTimestamp:2026-11-04 05:26:32 +0000 UTC,LastTimestamp:2026-11-04 05:26:32.22466038 +0000 UTC m=+0.669310078,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-185.us-west-2.compute.internal,}" Nov 04 05:26:33.059259 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:33.059185 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-185.us-west-2.compute.internal\" not found" Nov 04 05:26:33.160361 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:33.160330 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-185.us-west-2.compute.internal\" not found" Nov 04 05:26:33.191502 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:33.191468 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:26:33.192537 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:33.192517 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:26:33.192636 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:33.192550 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:26:33.192636 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:33.192560 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:26:33.236432 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:33.236392 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 05:21:32 +0000 UTC" deadline="2028-06-04 10:11:41.986372764 +0000 UTC" Nov 04 05:26:33.236432 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:33.236424 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13876h45m8.74995199s" Nov 04 05:26:33.260995 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:33.260972 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-185.us-west-2.compute.internal\" not found" Nov 04 05:26:33.362083 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:33.361978 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-185.us-west-2.compute.internal\" not found" Nov 04 05:26:33.463080 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:33.463029 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-185.us-west-2.compute.internal\" not found" Nov 04 05:26:33.536991 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:33.536965 2575 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:26:33.542451 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:33.542437 2575 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:26:33.610100 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:33.610074 2575 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:26:33.613144 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:33.613095 2575 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:26:33.681092 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:33.681065 2575 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:26:33.835166 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:33.835034 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eff7615010b2db89d82737c9bc2bd5b.slice/crio-77b95a272239a218daaf92e07e09e4b57be0328dc6885234f7911f53112e8631 WatchSource:0}: Error finding container 77b95a272239a218daaf92e07e09e4b57be0328dc6885234f7911f53112e8631: Status 404 returned error can't find the container with id 77b95a272239a218daaf92e07e09e4b57be0328dc6885234f7911f53112e8631 Nov 04 05:26:33.840056 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:33.840028 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbfe51fb7c8cecce5bf77e4c5e14a1c8.slice/crio-1e1ee7a2fb796112615a893a9293f3886537d412fe3873c3445751d9253ff591 WatchSource:0}: Error finding container 1e1ee7a2fb796112615a893a9293f3886537d412fe3873c3445751d9253ff591: Status 404 returned error can't find the container with id 1e1ee7a2fb796112615a893a9293f3886537d412fe3873c3445751d9253ff591 Nov 04 05:26:33.840804 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:33.840790 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 05:26:33.857514 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:33.857493 2575 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:26:33.957614 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:33.957536 2575 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:26:34.014804 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.014778 2575 apiserver.go:52] "Watching apiserver" Nov 04 05:26:34.041200 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.041170 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Nov 04 05:26:34.130702 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.130646 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" event={"ID":"dbfe51fb7c8cecce5bf77e4c5e14a1c8","Type":"ContainerStarted","Data":"1e1ee7a2fb796112615a893a9293f3886537d412fe3873c3445751d9253ff591"} Nov 04 05:26:34.131650 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.131625 2575 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Nov 04 05:26:34.131894 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.131871 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal" event={"ID":"4eff7615010b2db89d82737c9bc2bd5b","Type":"ContainerStarted","Data":"77b95a272239a218daaf92e07e09e4b57be0328dc6885234f7911f53112e8631"} Nov 04 05:26:34.132021 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.132002 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal","openshift-cluster-node-tuning-operator/tuned-dff2s","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal","openshift-ovn-kubernetes/ovnkube-node-cnn7v","kube-system/global-pull-secret-syncer-4pvss","openshift-dns/node-resolver-7b7zs","openshift-network-operator/iptables-alerter-mbh6p","openshift-multus/multus-nn7ng","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z","openshift-image-registry/node-ca-9bgz9","openshift-multus/multus-additional-cni-plugins-bzbgk","openshift-multus/network-metrics-daemon-chl9q","openshift-network-diagnostics/network-check-target-cmc7w","kube-system/konnectivity-agent-z2pjx"] Nov 04 05:26:34.136067 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.136051 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mbh6p" Nov 04 05:26:34.138246 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.138229 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:26:34.138339 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:34.138294 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmc7w" podUID="23cf3ade-4fac-44f1-813c-7f61427e2b83" Nov 04 05:26:34.140535 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.140521 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z2pjx" Nov 04 05:26:34.143003 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.142988 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.145463 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.145446 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:34.145546 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:34.145514 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4pvss" podUID="5a710da8-8925-4e16-b43d-fc06a999569b" Nov 04 05:26:34.147865 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.147845 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7b7zs" Nov 04 05:26:34.150435 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.150418 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.153633 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.153609 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.156208 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.156184 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.157663 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.157643 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.157755 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.157670 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e1fc5a00-d482-4aa6-955d-ce6ff425240e-env-overrides\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.157755 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.157687 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xftgr\" (UniqueName: \"kubernetes.io/projected/6c4e843c-e392-42ed-b842-508a67b0f64b-kube-api-access-xftgr\") pod \"iptables-alerter-mbh6p\" (UID: \"6c4e843c-e392-42ed-b842-508a67b0f64b\") " pod="openshift-network-operator/iptables-alerter-mbh6p" Nov 04 05:26:34.157755 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.157704 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/83db651d-42a9-48ba-a7ef-263e32d237d9-konnectivity-ca\") pod \"konnectivity-agent-z2pjx\" (UID: \"83db651d-42a9-48ba-a7ef-263e32d237d9\") " pod="kube-system/konnectivity-agent-z2pjx" Nov 04 05:26:34.157755 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.157727 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-var-lib-openvswitch\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.157936 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.157791 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-run-openvswitch\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.157936 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.157842 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e1fc5a00-d482-4aa6-955d-ce6ff425240e-ovnkube-config\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.157936 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.157878 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1fc5a00-d482-4aa6-955d-ce6ff425240e-ovn-node-metrics-cert\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.158101 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.157928 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e1fc5a00-d482-4aa6-955d-ce6ff425240e-ovnkube-script-lib\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.158101 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.157968 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-run-netns\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.158101 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.157997 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsqms\" (UniqueName: \"kubernetes.io/projected/e1fc5a00-d482-4aa6-955d-ce6ff425240e-kube-api-access-rsqms\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.158101 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.158024 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2bc5\" (UniqueName: \"kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5\") pod \"network-check-target-cmc7w\" (UID: \"23cf3ade-4fac-44f1-813c-7f61427e2b83\") " pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:26:34.158101 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.158072 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-systemd-units\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.158101 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.158099 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-log-socket\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.158383 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.158122 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/83db651d-42a9-48ba-a7ef-263e32d237d9-agent-certs\") pod \"konnectivity-agent-z2pjx\" (UID: \"83db651d-42a9-48ba-a7ef-263e32d237d9\") " pod="kube-system/konnectivity-agent-z2pjx" Nov 04 05:26:34.158383 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.158171 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6c4e843c-e392-42ed-b842-508a67b0f64b-iptables-alerter-script\") pod \"iptables-alerter-mbh6p\" (UID: \"6c4e843c-e392-42ed-b842-508a67b0f64b\") " pod="openshift-network-operator/iptables-alerter-mbh6p" Nov 04 05:26:34.158383 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.158209 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-slash\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.158383 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.158246 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-etc-openvswitch\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.158383 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.158288 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-kubelet\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.158383 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.158319 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-run-ovn\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.158383 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.158349 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-node-log\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.158383 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.158367 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-run-ovn-kubernetes\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.158383 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.158386 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-cni-bin\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.158762 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.158404 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-cni-netd\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.158762 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.158422 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c4e843c-e392-42ed-b842-508a67b0f64b-host-slash\") pod \"iptables-alerter-mbh6p\" (UID: \"6c4e843c-e392-42ed-b842-508a67b0f64b\") " pod="openshift-network-operator/iptables-alerter-mbh6p" Nov 04 05:26:34.158762 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.158475 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-run-systemd\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.158762 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.158693 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9bgz9" Nov 04 05:26:34.161163 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.161143 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.163456 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.163436 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:34.163532 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:34.163491 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-chl9q" podUID="b0e60cf0-1764-4c18-ab4e-da874feaff36" Nov 04 05:26:34.195518 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.195499 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:26:34.195610 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.195552 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Nov 04 05:26:34.195610 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.195554 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vw645\"" Nov 04 05:26:34.195610 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.195603 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Nov 04 05:26:34.199353 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.199332 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Nov 04 05:26:34.199429 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.199403 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Nov 04 05:26:34.199429 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.199417 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-8lkk9\"" Nov 04 05:26:34.201862 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.201848 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Nov 04 05:26:34.202005 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.201984 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Nov 04 05:26:34.202005 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.201990 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Nov 04 05:26:34.202203 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.202189 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Nov 04 05:26:34.202266 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.202193 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Nov 04 05:26:34.202327 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.202311 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-z6mlw\"" Nov 04 05:26:34.202415 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.202404 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Nov 04 05:26:34.206623 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.206606 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Nov 04 05:26:34.206725 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.206681 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Nov 04 05:26:34.206843 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.206831 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6nvfw\"" Nov 04 05:26:34.209243 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.209203 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Nov 04 05:26:34.209570 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.209547 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Nov 04 05:26:34.210045 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.210027 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Nov 04 05:26:34.210288 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.210274 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hqws2\"" Nov 04 05:26:34.212497 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.212478 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Nov 04 05:26:34.212575 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.212507 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Nov 04 05:26:34.212575 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.212509 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-x7j27\"" Nov 04 05:26:34.214119 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.214099 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Nov 04 05:26:34.214164 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.214151 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Nov 04 05:26:34.214977 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.214950 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Nov 04 05:26:34.215019 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.214949 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:26:34.215019 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.214966 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fz9sj\"" Nov 04 05:26:34.217344 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.217332 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Nov 04 05:26:34.217517 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.217503 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r57nx\"" Nov 04 05:26:34.244098 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.244066 2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Nov 04 05:26:34.258865 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.258841 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-run-openvswitch\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.258933 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.258878 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-etc-kubernetes\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.258974 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.258930 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-run-openvswitch\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.259009 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.258982 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5tw5\" (UniqueName: \"kubernetes.io/projected/ac2e1d23-6505-4936-9c05-70a76f8fd886-kube-api-access-f5tw5\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.259071 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259007 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-systemd\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.259071 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259023 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-cni-binary-copy\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.259159 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259091 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lhhb\" (UniqueName: \"kubernetes.io/projected/bf3a5689-535b-464f-9668-8d389d6396c5-kube-api-access-4lhhb\") pod \"node-resolver-7b7zs\" (UID: \"bf3a5689-535b-464f-9668-8d389d6396c5\") " pod="openshift-dns/node-resolver-7b7zs" Nov 04 05:26:34.259159 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259121 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsqms\" (UniqueName: \"kubernetes.io/projected/e1fc5a00-d482-4aa6-955d-ce6ff425240e-kube-api-access-rsqms\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.259159 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259143 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs\") pod \"network-metrics-daemon-chl9q\" (UID: \"b0e60cf0-1764-4c18-ab4e-da874feaff36\") " pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:34.259271 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259161 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-socket-dir-parent\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.259271 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259177 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-kubernetes\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.259271 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259199 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-sysctl-d\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.259271 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259214 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf3a5689-535b-464f-9668-8d389d6396c5-tmp-dir\") pod \"node-resolver-7b7zs\" (UID: \"bf3a5689-535b-464f-9668-8d389d6396c5\") " pod="openshift-dns/node-resolver-7b7zs" Nov 04 05:26:34.259271 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259230 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2bc5\" (UniqueName: \"kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5\") pod \"network-check-target-cmc7w\" (UID: \"23cf3ade-4fac-44f1-813c-7f61427e2b83\") " pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:26:34.259271 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259245 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-systemd-units\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.259271 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259260 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9df1fb81-e74f-45cb-b07a-026f323567b5-serviceca\") pod \"node-ca-9bgz9\" (UID: \"9df1fb81-e74f-45cb-b07a-026f323567b5\") " pod="openshift-image-registry/node-ca-9bgz9" Nov 04 05:26:34.259498 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259276 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.259498 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259302 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/83db651d-42a9-48ba-a7ef-263e32d237d9-agent-certs\") pod \"konnectivity-agent-z2pjx\" (UID: \"83db651d-42a9-48ba-a7ef-263e32d237d9\") " pod="kube-system/konnectivity-agent-z2pjx" Nov 04 05:26:34.259498 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259318 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-var-lib-kubelet\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.259498 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259334 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-conf-dir\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.259498 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259349 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-slash\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.259498 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259364 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-etc-openvswitch\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.259498 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259378 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-sys-fs\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.259498 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259393 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-kubelet\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.259498 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259409 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-run-ovn\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.259498 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259424 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-node-log\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.259498 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259444 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-cni-bin\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.259498 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259458 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-cni-netd\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.259498 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259481 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-lib-modules\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.259498 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259497 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-var-lib-kubelet\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.260062 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259512 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9df1fb81-e74f-45cb-b07a-026f323567b5-host\") pod \"node-ca-9bgz9\" (UID: \"9df1fb81-e74f-45cb-b07a-026f323567b5\") " pod="openshift-image-registry/node-ca-9bgz9" Nov 04 05:26:34.260062 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259528 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-run-systemd\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.260062 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259543 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e1fc5a00-d482-4aa6-955d-ce6ff425240e-env-overrides\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.260062 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259561 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-device-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.260062 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259576 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-cni-dir\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.260062 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259591 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-hostroot\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.260062 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259605 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-sysctl-conf\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.260062 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259622 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-sys\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.260062 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259636 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fc979bb-e663-4465-a2cc-16cb06be4963-tmp\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.260062 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259658 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xftgr\" (UniqueName: \"kubernetes.io/projected/6c4e843c-e392-42ed-b842-508a67b0f64b-kube-api-access-xftgr\") pod \"iptables-alerter-mbh6p\" (UID: \"6c4e843c-e392-42ed-b842-508a67b0f64b\") " pod="openshift-network-operator/iptables-alerter-mbh6p" Nov 04 05:26:34.260062 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259677 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/83db651d-42a9-48ba-a7ef-263e32d237d9-konnectivity-ca\") pod \"konnectivity-agent-z2pjx\" (UID: \"83db651d-42a9-48ba-a7ef-263e32d237d9\") " pod="kube-system/konnectivity-agent-z2pjx" Nov 04 05:26:34.260062 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259700 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-system-cni-dir\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.260062 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259715 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-tuned\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.260062 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259730 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5a710da8-8925-4e16-b43d-fc06a999569b-dbus\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:34.260062 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259748 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:34.260062 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259764 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e1fc5a00-d482-4aa6-955d-ce6ff425240e-ovnkube-config\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.260062 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259779 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1fc5a00-d482-4aa6-955d-ce6ff425240e-ovn-node-metrics-cert\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.260815 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259795 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-run-netns\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.260815 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259819 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-run\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.260815 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259855 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hwxg\" (UniqueName: \"kubernetes.io/projected/9df1fb81-e74f-45cb-b07a-026f323567b5-kube-api-access-8hwxg\") pod \"node-ca-9bgz9\" (UID: \"9df1fb81-e74f-45cb-b07a-026f323567b5\") " pod="openshift-image-registry/node-ca-9bgz9" Nov 04 05:26:34.260815 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259860 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-cni-bin\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.260815 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259918 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-node-log\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.260815 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259955 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-kubelet\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.260815 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259971 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-etc-openvswitch\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.260815 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.259987 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-systemd-units\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.260815 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260033 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-slash\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.260815 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260088 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e1fc5a00-d482-4aa6-955d-ce6ff425240e-ovnkube-script-lib\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.260815 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260135 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-run-netns\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.260815 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260128 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-run-systemd\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.260815 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260166 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpqzl\" (UniqueName: \"kubernetes.io/projected/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-kube-api-access-mpqzl\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.260815 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260196 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-run-k8s-cni-cncf-io\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.260815 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260203 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-cni-netd\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.260815 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260222 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-sysconfig\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.260815 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260283 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-host\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.261479 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260292 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-run-netns\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.261479 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260321 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzngz\" (UniqueName: \"kubernetes.io/projected/4fc979bb-e663-4465-a2cc-16cb06be4963-kube-api-access-nzngz\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.261479 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260307 2575 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Nov 04 05:26:34.261479 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260353 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-system-cni-dir\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.261479 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260385 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-log-socket\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.261479 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260447 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-cnibin\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.261479 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260475 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-etc-selinux\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.261479 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260507 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nptl\" (UniqueName: \"kubernetes.io/projected/b0e60cf0-1764-4c18-ab4e-da874feaff36-kube-api-access-4nptl\") pod \"network-metrics-daemon-chl9q\" (UID: \"b0e60cf0-1764-4c18-ab4e-da874feaff36\") " pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:34.261479 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260527 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-run-ovn\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.261479 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260533 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ac2e1d23-6505-4936-9c05-70a76f8fd886-cni-binary-copy\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.261479 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260566 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-var-lib-cni-bin\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.261479 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260609 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.261479 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260611 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-log-socket\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.261479 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260627 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-daemon-config\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.261479 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260684 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6c4e843c-e392-42ed-b842-508a67b0f64b-iptables-alerter-script\") pod \"iptables-alerter-mbh6p\" (UID: \"6c4e843c-e392-42ed-b842-508a67b0f64b\") " pod="openshift-network-operator/iptables-alerter-mbh6p" Nov 04 05:26:34.261479 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260714 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.261479 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260731 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-socket-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.262026 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260792 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.262026 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260822 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-run-ovn-kubernetes\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.262026 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260863 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-var-lib-cni-multus\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.262026 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260882 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-run-ovn-kubernetes\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.262026 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260893 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-cnibin\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.262026 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260932 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vklnx\" (UniqueName: \"kubernetes.io/projected/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-kube-api-access-vklnx\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.262026 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260962 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bf3a5689-535b-464f-9668-8d389d6396c5-hosts-file\") pod \"node-resolver-7b7zs\" (UID: \"bf3a5689-535b-464f-9668-8d389d6396c5\") " pod="openshift-dns/node-resolver-7b7zs" Nov 04 05:26:34.262026 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.260987 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c4e843c-e392-42ed-b842-508a67b0f64b-host-slash\") pod \"iptables-alerter-mbh6p\" (UID: \"6c4e843c-e392-42ed-b842-508a67b0f64b\") " pod="openshift-network-operator/iptables-alerter-mbh6p" Nov 04 05:26:34.262026 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.261018 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.262026 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.261062 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c4e843c-e392-42ed-b842-508a67b0f64b-host-slash\") pod \"iptables-alerter-mbh6p\" (UID: \"6c4e843c-e392-42ed-b842-508a67b0f64b\") " pod="openshift-network-operator/iptables-alerter-mbh6p" Nov 04 05:26:34.262026 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.261116 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-modprobe-d\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.262026 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.261158 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-var-lib-openvswitch\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.262026 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.261211 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-registration-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.262026 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.261223 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.262026 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.261245 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-run-multus-certs\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.262026 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.261276 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-var-lib-openvswitch\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.262544 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.261322 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e1fc5a00-d482-4aa6-955d-ce6ff425240e-env-overrides\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.262544 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.261339 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5a710da8-8925-4e16-b43d-fc06a999569b-kubelet-config\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:34.262544 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.261377 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e1fc5a00-d482-4aa6-955d-ce6ff425240e-ovnkube-config\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.262544 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.261393 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/83db651d-42a9-48ba-a7ef-263e32d237d9-konnectivity-ca\") pod \"konnectivity-agent-z2pjx\" (UID: \"83db651d-42a9-48ba-a7ef-263e32d237d9\") " pod="kube-system/konnectivity-agent-z2pjx" Nov 04 05:26:34.262544 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.261637 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e1fc5a00-d482-4aa6-955d-ce6ff425240e-ovnkube-script-lib\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.262544 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.261855 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6c4e843c-e392-42ed-b842-508a67b0f64b-iptables-alerter-script\") pod \"iptables-alerter-mbh6p\" (UID: \"6c4e843c-e392-42ed-b842-508a67b0f64b\") " pod="openshift-network-operator/iptables-alerter-mbh6p" Nov 04 05:26:34.263570 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.263535 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1fc5a00-d482-4aa6-955d-ce6ff425240e-ovn-node-metrics-cert\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.263652 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.263637 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/83db651d-42a9-48ba-a7ef-263e32d237d9-agent-certs\") pod \"konnectivity-agent-z2pjx\" (UID: \"83db651d-42a9-48ba-a7ef-263e32d237d9\") " pod="kube-system/konnectivity-agent-z2pjx" Nov 04 05:26:34.299156 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.299132 2575 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Nov 04 05:26:34.299247 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.299163 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fl7wt" Nov 04 05:26:34.303812 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.303785 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Nov 04 05:26:34.303881 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.303840 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Nov 04 05:26:34.303881 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.303863 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Nov 04 05:26:34.304006 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.303908 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Nov 04 05:26:34.304041 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.304019 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qgv9m\"" Nov 04 05:26:34.326269 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:34.326194 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:26:34.326269 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:34.326229 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:26:34.326269 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:34.326246 2575 projected.go:196] Error preparing data for projected volume kube-api-access-r2bc5 for pod openshift-network-diagnostics/network-check-target-cmc7w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:34.326567 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:34.326367 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5 podName:23cf3ade-4fac-44f1-813c-7f61427e2b83 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:34.826310832 +0000 UTC m=+3.270960545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-r2bc5" (UniqueName: "kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5") pod "network-check-target-cmc7w" (UID: "23cf3ade-4fac-44f1-813c-7f61427e2b83") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:34.329249 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.329223 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xftgr\" (UniqueName: \"kubernetes.io/projected/6c4e843c-e392-42ed-b842-508a67b0f64b-kube-api-access-xftgr\") pod \"iptables-alerter-mbh6p\" (UID: \"6c4e843c-e392-42ed-b842-508a67b0f64b\") " pod="openshift-network-operator/iptables-alerter-mbh6p" Nov 04 05:26:34.329368 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.329261 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsqms\" (UniqueName: \"kubernetes.io/projected/e1fc5a00-d482-4aa6-955d-ce6ff425240e-kube-api-access-rsqms\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.362196 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362165 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-lib-modules\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.362294 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362197 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-var-lib-kubelet\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.362294 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362213 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9df1fb81-e74f-45cb-b07a-026f323567b5-host\") pod \"node-ca-9bgz9\" (UID: \"9df1fb81-e74f-45cb-b07a-026f323567b5\") " pod="openshift-image-registry/node-ca-9bgz9" Nov 04 05:26:34.362294 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362231 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-device-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.362294 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362249 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-cni-dir\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.362294 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362271 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-hostroot\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.362294 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362293 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-sysctl-conf\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.362587 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362313 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-sys\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.362587 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362313 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-var-lib-kubelet\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.362587 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362336 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fc979bb-e663-4465-a2cc-16cb06be4963-tmp\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.362587 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362344 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-device-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.362587 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362367 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-system-cni-dir\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.362587 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362383 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-lib-modules\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.362587 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362393 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-tuned\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.362587 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362363 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-hostroot\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.362587 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362414 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-sys\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.362587 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362452 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-system-cni-dir\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.362587 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362417 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5a710da8-8925-4e16-b43d-fc06a999569b-dbus\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:34.362587 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362462 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-sysctl-conf\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.362587 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362483 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:34.362587 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362489 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-cni-dir\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.362587 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362495 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5a710da8-8925-4e16-b43d-fc06a999569b-dbus\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:34.362587 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362538 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-run-netns\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.362587 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:34.362577 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:34.363440 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362616 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-run\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.363440 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:34.362646 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret podName:5a710da8-8925-4e16-b43d-fc06a999569b nodeName:}" failed. No retries permitted until 2026-11-04 05:26:34.862627144 +0000 UTC m=+3.307276853 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret") pod "global-pull-secret-syncer-4pvss" (UID: "5a710da8-8925-4e16-b43d-fc06a999569b") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:34.363440 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362661 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-run\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.363440 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362672 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-run-netns\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.363440 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362678 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hwxg\" (UniqueName: \"kubernetes.io/projected/9df1fb81-e74f-45cb-b07a-026f323567b5-kube-api-access-8hwxg\") pod \"node-ca-9bgz9\" (UID: \"9df1fb81-e74f-45cb-b07a-026f323567b5\") " pod="openshift-image-registry/node-ca-9bgz9" Nov 04 05:26:34.363440 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362691 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9df1fb81-e74f-45cb-b07a-026f323567b5-host\") pod \"node-ca-9bgz9\" (UID: \"9df1fb81-e74f-45cb-b07a-026f323567b5\") " pod="openshift-image-registry/node-ca-9bgz9" Nov 04 05:26:34.363440 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362717 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpqzl\" (UniqueName: \"kubernetes.io/projected/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-kube-api-access-mpqzl\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.363440 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362740 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-run-k8s-cni-cncf-io\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.363440 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362756 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-sysconfig\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.363440 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362773 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-host\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.363440 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362796 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzngz\" (UniqueName: \"kubernetes.io/projected/4fc979bb-e663-4465-a2cc-16cb06be4963-kube-api-access-nzngz\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.363440 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362821 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-system-cni-dir\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.363440 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362839 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-run-k8s-cni-cncf-io\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.363440 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362901 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-system-cni-dir\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.363440 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362847 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-sysconfig\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.363440 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362906 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-cnibin\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.363440 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362847 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-cnibin\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.364350 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362967 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-etc-selinux\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.364350 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362985 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nptl\" (UniqueName: \"kubernetes.io/projected/b0e60cf0-1764-4c18-ab4e-da874feaff36-kube-api-access-4nptl\") pod \"network-metrics-daemon-chl9q\" (UID: \"b0e60cf0-1764-4c18-ab4e-da874feaff36\") " pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:34.364350 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.362905 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-host\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.364350 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363009 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ac2e1d23-6505-4936-9c05-70a76f8fd886-cni-binary-copy\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.364350 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363035 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-var-lib-cni-bin\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.364350 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363072 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-etc-selinux\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.364350 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363095 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-var-lib-cni-bin\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.364350 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363109 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.364350 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363142 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-daemon-config\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.364350 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363188 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.364350 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363213 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-socket-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.364350 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363237 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.364350 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363255 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.364350 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363266 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-var-lib-cni-multus\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.364350 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363276 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.364350 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363291 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-cnibin\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.364350 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363317 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vklnx\" (UniqueName: \"kubernetes.io/projected/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-kube-api-access-vklnx\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.365025 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363344 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bf3a5689-535b-464f-9668-8d389d6396c5-hosts-file\") pod \"node-resolver-7b7zs\" (UID: \"bf3a5689-535b-464f-9668-8d389d6396c5\") " pod="openshift-dns/node-resolver-7b7zs" Nov 04 05:26:34.365025 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363360 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-socket-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.365025 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363378 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-modprobe-d\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.365025 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363404 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-registration-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.365025 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363412 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-cnibin\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.365025 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363429 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-run-multus-certs\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.365025 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363456 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5a710da8-8925-4e16-b43d-fc06a999569b-kubelet-config\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:34.365025 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363486 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-etc-kubernetes\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.365025 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363512 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5tw5\" (UniqueName: \"kubernetes.io/projected/ac2e1d23-6505-4936-9c05-70a76f8fd886-kube-api-access-f5tw5\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.365025 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363541 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-systemd\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.365025 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363568 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-cni-binary-copy\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.365025 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363592 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lhhb\" (UniqueName: \"kubernetes.io/projected/bf3a5689-535b-464f-9668-8d389d6396c5-kube-api-access-4lhhb\") pod \"node-resolver-7b7zs\" (UID: \"bf3a5689-535b-464f-9668-8d389d6396c5\") " pod="openshift-dns/node-resolver-7b7zs" Nov 04 05:26:34.365025 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363611 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs\") pod \"network-metrics-daemon-chl9q\" (UID: \"b0e60cf0-1764-4c18-ab4e-da874feaff36\") " pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:34.365025 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363628 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-socket-dir-parent\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.365025 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363652 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-kubernetes\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.365025 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363669 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-etc-kubernetes\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.365025 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363458 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-var-lib-cni-multus\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.365565 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363698 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5a710da8-8925-4e16-b43d-fc06a999569b-kubelet-config\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:34.365565 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363724 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-socket-dir-parent\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.365565 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363754 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-registration-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.365565 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363772 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-modprobe-d\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.365565 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363677 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-sysctl-d\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.365565 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363802 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-run-multus-certs\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.365565 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363812 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-systemd\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.365565 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363820 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-sysctl-d\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.365565 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363628 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bf3a5689-535b-464f-9668-8d389d6396c5-hosts-file\") pod \"node-resolver-7b7zs\" (UID: \"bf3a5689-535b-464f-9668-8d389d6396c5\") " pod="openshift-dns/node-resolver-7b7zs" Nov 04 05:26:34.365565 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363813 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf3a5689-535b-464f-9668-8d389d6396c5-tmp-dir\") pod \"node-resolver-7b7zs\" (UID: \"bf3a5689-535b-464f-9668-8d389d6396c5\") " pod="openshift-dns/node-resolver-7b7zs" Nov 04 05:26:34.365565 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363888 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9df1fb81-e74f-45cb-b07a-026f323567b5-serviceca\") pod \"node-ca-9bgz9\" (UID: \"9df1fb81-e74f-45cb-b07a-026f323567b5\") " pod="openshift-image-registry/node-ca-9bgz9" Nov 04 05:26:34.365565 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363917 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.365565 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:34.363944 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:34.365565 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363981 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-var-lib-kubelet\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.365565 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:34.363998 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs podName:b0e60cf0-1764-4c18-ab4e-da874feaff36 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:34.863982294 +0000 UTC m=+3.308631980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs") pod "network-metrics-daemon-chl9q" (UID: "b0e60cf0-1764-4c18-ab4e-da874feaff36") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:34.365565 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.363944 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-var-lib-kubelet\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.365565 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.364027 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-kubernetes\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.366050 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.364043 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-conf-dir\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.366050 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.364075 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-sys-fs\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.366050 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.364181 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-sys-fs\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.366050 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.364190 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-conf-dir\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.366050 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.364274 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ac2e1d23-6505-4936-9c05-70a76f8fd886-cni-binary-copy\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.366050 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.364405 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-daemon-config\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.366050 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.364441 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf3a5689-535b-464f-9668-8d389d6396c5-tmp-dir\") pod \"node-resolver-7b7zs\" (UID: \"bf3a5689-535b-464f-9668-8d389d6396c5\") " pod="openshift-dns/node-resolver-7b7zs" Nov 04 05:26:34.366050 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.364547 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.366050 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.364899 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9df1fb81-e74f-45cb-b07a-026f323567b5-serviceca\") pod \"node-ca-9bgz9\" (UID: \"9df1fb81-e74f-45cb-b07a-026f323567b5\") " pod="openshift-image-registry/node-ca-9bgz9" Nov 04 05:26:34.366050 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.364905 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-cni-binary-copy\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.366050 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.364926 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-tuned\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.366050 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.364956 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fc979bb-e663-4465-a2cc-16cb06be4963-tmp\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.366050 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.364975 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.431731 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.431691 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hwxg\" (UniqueName: \"kubernetes.io/projected/9df1fb81-e74f-45cb-b07a-026f323567b5-kube-api-access-8hwxg\") pod \"node-ca-9bgz9\" (UID: \"9df1fb81-e74f-45cb-b07a-026f323567b5\") " pod="openshift-image-registry/node-ca-9bgz9" Nov 04 05:26:34.432165 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.432137 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5tw5\" (UniqueName: \"kubernetes.io/projected/ac2e1d23-6505-4936-9c05-70a76f8fd886-kube-api-access-f5tw5\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.432551 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.432527 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpqzl\" (UniqueName: \"kubernetes.io/projected/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-kube-api-access-mpqzl\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.432909 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.432881 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nptl\" (UniqueName: \"kubernetes.io/projected/b0e60cf0-1764-4c18-ab4e-da874feaff36-kube-api-access-4nptl\") pod \"network-metrics-daemon-chl9q\" (UID: \"b0e60cf0-1764-4c18-ab4e-da874feaff36\") " pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:34.433000 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.432972 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vklnx\" (UniqueName: \"kubernetes.io/projected/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-kube-api-access-vklnx\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.433048 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.433031 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lhhb\" (UniqueName: \"kubernetes.io/projected/bf3a5689-535b-464f-9668-8d389d6396c5-kube-api-access-4lhhb\") pod \"node-resolver-7b7zs\" (UID: \"bf3a5689-535b-464f-9668-8d389d6396c5\") " pod="openshift-dns/node-resolver-7b7zs" Nov 04 05:26:34.433198 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.433182 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzngz\" (UniqueName: \"kubernetes.io/projected/4fc979bb-e663-4465-a2cc-16cb06be4963-kube-api-access-nzngz\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.449522 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.449504 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mbh6p" Nov 04 05:26:34.456211 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.456195 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z2pjx" Nov 04 05:26:34.459816 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:34.459775 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c4e843c_e392_42ed_b842_508a67b0f64b.slice/crio-e456f1812cccb3da24cd453c17ed3fc2f10fe153298688919214022cc2035c00 WatchSource:0}: Error finding container e456f1812cccb3da24cd453c17ed3fc2f10fe153298688919214022cc2035c00: Status 404 returned error can't find the container with id e456f1812cccb3da24cd453c17ed3fc2f10fe153298688919214022cc2035c00 Nov 04 05:26:34.465573 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:34.465550 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83db651d_42a9_48ba_a7ef_263e32d237d9.slice/crio-e06cf59f7386c12d3abbe5721afac2d9bb4ec36682b2747705eb514275347b6d WatchSource:0}: Error finding container e06cf59f7386c12d3abbe5721afac2d9bb4ec36682b2747705eb514275347b6d: Status 404 returned error can't find the container with id e06cf59f7386c12d3abbe5721afac2d9bb4ec36682b2747705eb514275347b6d Nov 04 05:26:34.475055 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.475026 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7b7zs" Nov 04 05:26:34.483968 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.483944 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:26:34.484870 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:34.484843 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf3a5689_535b_464f_9668_8d389d6396c5.slice/crio-7dac931844b45988d66af94c90834d70aa8f08c60d881ddb099b21a1b925e5a0 WatchSource:0}: Error finding container 7dac931844b45988d66af94c90834d70aa8f08c60d881ddb099b21a1b925e5a0: Status 404 returned error can't find the container with id 7dac931844b45988d66af94c90834d70aa8f08c60d881ddb099b21a1b925e5a0 Nov 04 05:26:34.486397 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.486374 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nn7ng" Nov 04 05:26:34.489163 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.489036 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:34.491341 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.491322 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:26:34.493414 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.493391 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9bgz9" Nov 04 05:26:34.495355 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.495338 2575 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:26:34.497212 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:34.497191 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ca23bb0_8c76_4d96_8ab5_4b57dbcb1050.slice/crio-9d880bbb709757092439f940aec40765b05af6049d4904cba4b6734b167190a0 WatchSource:0}: Error finding container 9d880bbb709757092439f940aec40765b05af6049d4904cba4b6734b167190a0: Status 404 returned error can't find the container with id 9d880bbb709757092439f940aec40765b05af6049d4904cba4b6734b167190a0 Nov 04 05:26:34.498227 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:34.498211 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac2e1d23_6505_4936_9c05_70a76f8fd886.slice/crio-7dd94360c70bdb8282c2497a257293d5cc50c73f8a0c7335f4f731319dac7186 WatchSource:0}: Error finding container 7dd94360c70bdb8282c2497a257293d5cc50c73f8a0c7335f4f731319dac7186: Status 404 returned error can't find the container with id 7dd94360c70bdb8282c2497a257293d5cc50c73f8a0c7335f4f731319dac7186 Nov 04 05:26:34.503511 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:34.503486 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1fc5a00_d482_4aa6_955d_ce6ff425240e.slice/crio-32c7911aad73a69827aa71e308653a4a1881fca1a866a239e682ee0fb082b3b8 WatchSource:0}: Error finding container 32c7911aad73a69827aa71e308653a4a1881fca1a866a239e682ee0fb082b3b8: Status 404 returned error can't find the container with id 32c7911aad73a69827aa71e308653a4a1881fca1a866a239e682ee0fb082b3b8 Nov 04 05:26:34.504405 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:34.504375 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fc979bb_e663_4465_a2cc_16cb06be4963.slice/crio-1f232731bfd578d17c00aebd39bb9830fffc9dab18bb58ba6d855dd4358ef629 WatchSource:0}: Error finding container 1f232731bfd578d17c00aebd39bb9830fffc9dab18bb58ba6d855dd4358ef629: Status 404 returned error can't find the container with id 1f232731bfd578d17c00aebd39bb9830fffc9dab18bb58ba6d855dd4358ef629 Nov 04 05:26:34.506565 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:34.506530 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9df1fb81_e74f_45cb_b07a_026f323567b5.slice/crio-24cdb6fe97a2013caf78391931390bf69ab3ce4a9eaba0a900ce8608f2503d69 WatchSource:0}: Error finding container 24cdb6fe97a2013caf78391931390bf69ab3ce4a9eaba0a900ce8608f2503d69: Status 404 returned error can't find the container with id 24cdb6fe97a2013caf78391931390bf69ab3ce4a9eaba0a900ce8608f2503d69 Nov 04 05:26:34.510194 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.510179 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:26:34.519309 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:26:34.519290 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99a606bd_b4a1_43e9_8525_a7afcd1eae0f.slice/crio-2a9d4ef75ba4781c64c7c7c60ddf779f1b146f6a366f618c4b12bdd284719c10 WatchSource:0}: Error finding container 2a9d4ef75ba4781c64c7c7c60ddf779f1b146f6a366f618c4b12bdd284719c10: Status 404 returned error can't find the container with id 2a9d4ef75ba4781c64c7c7c60ddf779f1b146f6a366f618c4b12bdd284719c10 Nov 04 05:26:34.868149 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.868060 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs\") pod \"network-metrics-daemon-chl9q\" (UID: \"b0e60cf0-1764-4c18-ab4e-da874feaff36\") " pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:34.868149 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.868098 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2bc5\" (UniqueName: \"kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5\") pod \"network-check-target-cmc7w\" (UID: \"23cf3ade-4fac-44f1-813c-7f61427e2b83\") " pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:26:34.868149 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:34.868126 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:34.868375 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:34.868213 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:34.868375 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:34.868215 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:34.868375 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:34.868234 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:26:34.868375 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:34.868258 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:26:34.868375 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:34.868267 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret podName:5a710da8-8925-4e16-b43d-fc06a999569b nodeName:}" failed. No retries permitted until 2026-11-04 05:26:35.868253862 +0000 UTC m=+4.312903562 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret") pod "global-pull-secret-syncer-4pvss" (UID: "5a710da8-8925-4e16-b43d-fc06a999569b") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:34.868375 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:34.868271 2575 projected.go:196] Error preparing data for projected volume kube-api-access-r2bc5 for pod openshift-network-diagnostics/network-check-target-cmc7w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:34.868375 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:34.868281 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs podName:b0e60cf0-1764-4c18-ab4e-da874feaff36 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:35.868274639 +0000 UTC m=+4.312924324 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs") pod "network-metrics-daemon-chl9q" (UID: "b0e60cf0-1764-4c18-ab4e-da874feaff36") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:34.868375 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:34.868309 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5 podName:23cf3ade-4fac-44f1-813c-7f61427e2b83 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:35.868296609 +0000 UTC m=+4.312946318 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-r2bc5" (UniqueName: "kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5") pod "network-check-target-cmc7w" (UID: "23cf3ade-4fac-44f1-813c-7f61427e2b83") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:35.135085 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:35.134986 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzbgk" event={"ID":"99a606bd-b4a1-43e9-8525-a7afcd1eae0f","Type":"ContainerStarted","Data":"2a9d4ef75ba4781c64c7c7c60ddf779f1b146f6a366f618c4b12bdd284719c10"} Nov 04 05:26:35.136667 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:35.136635 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nn7ng" event={"ID":"ac2e1d23-6505-4936-9c05-70a76f8fd886","Type":"ContainerStarted","Data":"7dd94360c70bdb8282c2497a257293d5cc50c73f8a0c7335f4f731319dac7186"} Nov 04 05:26:35.137875 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:35.137841 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9bgz9" event={"ID":"9df1fb81-e74f-45cb-b07a-026f323567b5","Type":"ContainerStarted","Data":"24cdb6fe97a2013caf78391931390bf69ab3ce4a9eaba0a900ce8608f2503d69"} Nov 04 05:26:35.139309 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:35.139258 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dff2s" event={"ID":"4fc979bb-e663-4465-a2cc-16cb06be4963","Type":"ContainerStarted","Data":"1f232731bfd578d17c00aebd39bb9830fffc9dab18bb58ba6d855dd4358ef629"} Nov 04 05:26:35.140473 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:35.140451 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" event={"ID":"e1fc5a00-d482-4aa6-955d-ce6ff425240e","Type":"ContainerStarted","Data":"32c7911aad73a69827aa71e308653a4a1881fca1a866a239e682ee0fb082b3b8"} Nov 04 05:26:35.141641 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:35.141609 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" event={"ID":"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050","Type":"ContainerStarted","Data":"9d880bbb709757092439f940aec40765b05af6049d4904cba4b6734b167190a0"} Nov 04 05:26:35.146128 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:35.143192 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7b7zs" event={"ID":"bf3a5689-535b-464f-9668-8d389d6396c5","Type":"ContainerStarted","Data":"7dac931844b45988d66af94c90834d70aa8f08c60d881ddb099b21a1b925e5a0"} Nov 04 05:26:35.146128 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:35.144490 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z2pjx" event={"ID":"83db651d-42a9-48ba-a7ef-263e32d237d9","Type":"ContainerStarted","Data":"e06cf59f7386c12d3abbe5721afac2d9bb4ec36682b2747705eb514275347b6d"} Nov 04 05:26:35.146128 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:35.145823 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mbh6p" event={"ID":"6c4e843c-e392-42ed-b842-508a67b0f64b","Type":"ContainerStarted","Data":"e456f1812cccb3da24cd453c17ed3fc2f10fe153298688919214022cc2035c00"} Nov 04 05:26:35.303494 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:35.303439 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 05:21:34 +0000 UTC" deadline="2028-04-15 23:19:13.902050582 +0000 UTC" Nov 04 05:26:35.303494 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:35.303484 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12689h52m38.598569997s" Nov 04 05:26:35.877362 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:35.877322 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs\") pod \"network-metrics-daemon-chl9q\" (UID: \"b0e60cf0-1764-4c18-ab4e-da874feaff36\") " pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:35.877541 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:35.877378 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2bc5\" (UniqueName: \"kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5\") pod \"network-check-target-cmc7w\" (UID: \"23cf3ade-4fac-44f1-813c-7f61427e2b83\") " pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:26:35.877541 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:35.877427 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:35.877673 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:35.877568 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:35.877673 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:35.877633 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret podName:5a710da8-8925-4e16-b43d-fc06a999569b nodeName:}" failed. No retries permitted until 2026-11-04 05:26:37.877613916 +0000 UTC m=+6.322263617 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret") pod "global-pull-secret-syncer-4pvss" (UID: "5a710da8-8925-4e16-b43d-fc06a999569b") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:35.877780 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:35.877700 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:35.877780 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:35.877732 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs podName:b0e60cf0-1764-4c18-ab4e-da874feaff36 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:37.877720544 +0000 UTC m=+6.322370244 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs") pod "network-metrics-daemon-chl9q" (UID: "b0e60cf0-1764-4c18-ab4e-da874feaff36") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:35.877890 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:35.877810 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:26:35.877890 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:35.877829 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:26:35.877890 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:35.877842 2575 projected.go:196] Error preparing data for projected volume kube-api-access-r2bc5 for pod openshift-network-diagnostics/network-check-target-cmc7w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:35.877890 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:35.877877 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5 podName:23cf3ade-4fac-44f1-813c-7f61427e2b83 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:37.877866132 +0000 UTC m=+6.322515840 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-r2bc5" (UniqueName: "kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5") pod "network-check-target-cmc7w" (UID: "23cf3ade-4fac-44f1-813c-7f61427e2b83") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:36.124413 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:36.124380 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:36.124598 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:36.124516 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4pvss" podUID="5a710da8-8925-4e16-b43d-fc06a999569b" Nov 04 05:26:36.124676 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:36.124615 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:36.124729 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:36.124710 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-chl9q" podUID="b0e60cf0-1764-4c18-ab4e-da874feaff36" Nov 04 05:26:36.130262 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:36.127159 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:26:36.130262 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:36.127276 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmc7w" podUID="23cf3ade-4fac-44f1-813c-7f61427e2b83" Nov 04 05:26:36.304185 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:36.304072 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 05:21:34 +0000 UTC" deadline="2028-07-03 08:12:43.224271735 +0000 UTC" Nov 04 05:26:36.304185 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:36.304117 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14570h46m6.920160107s" Nov 04 05:26:37.895356 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:37.895305 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs\") pod \"network-metrics-daemon-chl9q\" (UID: \"b0e60cf0-1764-4c18-ab4e-da874feaff36\") " pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:37.895896 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:37.895370 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2bc5\" (UniqueName: \"kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5\") pod \"network-check-target-cmc7w\" (UID: \"23cf3ade-4fac-44f1-813c-7f61427e2b83\") " pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:26:37.895896 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:37.895434 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:37.895896 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:37.895616 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:37.895896 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:37.895698 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret podName:5a710da8-8925-4e16-b43d-fc06a999569b nodeName:}" failed. No retries permitted until 2026-11-04 05:26:41.895675341 +0000 UTC m=+10.340325048 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret") pod "global-pull-secret-syncer-4pvss" (UID: "5a710da8-8925-4e16-b43d-fc06a999569b") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:37.895896 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:37.895795 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:37.895896 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:37.895833 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs podName:b0e60cf0-1764-4c18-ab4e-da874feaff36 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:41.89582106 +0000 UTC m=+10.340470762 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs") pod "network-metrics-daemon-chl9q" (UID: "b0e60cf0-1764-4c18-ab4e-da874feaff36") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:37.896223 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:37.895925 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:26:37.896223 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:37.895950 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:26:37.896223 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:37.895964 2575 projected.go:196] Error preparing data for projected volume kube-api-access-r2bc5 for pod openshift-network-diagnostics/network-check-target-cmc7w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:37.896223 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:37.896004 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5 podName:23cf3ade-4fac-44f1-813c-7f61427e2b83 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:41.895991301 +0000 UTC m=+10.340641003 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-r2bc5" (UniqueName: "kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5") pod "network-check-target-cmc7w" (UID: "23cf3ade-4fac-44f1-813c-7f61427e2b83") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:38.125162 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:38.125126 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:26:38.125403 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:38.125258 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmc7w" podUID="23cf3ade-4fac-44f1-813c-7f61427e2b83" Nov 04 05:26:38.125403 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:38.125322 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:38.125403 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:38.125394 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4pvss" podUID="5a710da8-8925-4e16-b43d-fc06a999569b" Nov 04 05:26:38.125582 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:38.125533 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:38.125633 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:38.125619 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-chl9q" podUID="b0e60cf0-1764-4c18-ab4e-da874feaff36" Nov 04 05:26:39.175940 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:39.175708 2575 generic.go:356] "Generic (PLEG): container finished" podID="dbfe51fb7c8cecce5bf77e4c5e14a1c8" containerID="da6eab7047edd1b7ec50cf1e1c266ef9d31f80a89f6149aca970c0f1ee6d6570" exitCode=0 Nov 04 05:26:39.176351 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:39.175912 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" event={"ID":"dbfe51fb7c8cecce5bf77e4c5e14a1c8","Type":"ContainerDied","Data":"da6eab7047edd1b7ec50cf1e1c266ef9d31f80a89f6149aca970c0f1ee6d6570"} Nov 04 05:26:39.181861 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:39.181242 2575 generic.go:356] "Generic (PLEG): container finished" podID="99a606bd-b4a1-43e9-8525-a7afcd1eae0f" containerID="f8b96a7b6326eec6f3ec6c663fef9b72a6f1752322ffafbc324896d894e7e265" exitCode=0 Nov 04 05:26:39.181861 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:39.181290 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzbgk" event={"ID":"99a606bd-b4a1-43e9-8525-a7afcd1eae0f","Type":"ContainerDied","Data":"f8b96a7b6326eec6f3ec6c663fef9b72a6f1752322ffafbc324896d894e7e265"} Nov 04 05:26:40.126255 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:40.126154 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:40.126399 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:40.126353 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-chl9q" podUID="b0e60cf0-1764-4c18-ab4e-da874feaff36" Nov 04 05:26:40.126486 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:40.126435 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:26:40.126543 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:40.126504 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmc7w" podUID="23cf3ade-4fac-44f1-813c-7f61427e2b83" Nov 04 05:26:40.126598 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:40.126560 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:40.126651 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:40.126629 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4pvss" podUID="5a710da8-8925-4e16-b43d-fc06a999569b" Nov 04 05:26:40.196608 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:40.195905 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal" event={"ID":"4eff7615010b2db89d82737c9bc2bd5b","Type":"ContainerStarted","Data":"2e9534c89fb0f5d92c2f49d879e10cda28184840ebd40bffa99a0fa3225b7fc4"} Nov 04 05:26:40.199978 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:40.199525 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z2pjx" event={"ID":"83db651d-42a9-48ba-a7ef-263e32d237d9","Type":"ContainerStarted","Data":"43d66807791d8cc45f1b61b48cc6284b71e6bedb430bd41b0457ded4122225ac"} Nov 04 05:26:40.202708 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:40.202275 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" event={"ID":"dbfe51fb7c8cecce5bf77e4c5e14a1c8","Type":"ContainerStarted","Data":"71a61d8a6e10b4b328e4875b2c310a169021170d5fee56d3f292c1031cc5bf72"} Nov 04 05:26:40.316093 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:40.315541 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal" podStartSLOduration=7.31552377 podStartE2EDuration="7.31552377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:40.314964636 +0000 UTC m=+8.759614346" watchObservedRunningTime="2026-11-04 05:26:40.31552377 +0000 UTC m=+8.760173477" Nov 04 05:26:40.555516 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:40.555454 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" podStartSLOduration=7.555434509 podStartE2EDuration="7.555434509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:40.435174405 +0000 UTC m=+8.879824114" watchObservedRunningTime="2026-11-04 05:26:40.555434509 +0000 UTC m=+9.000084215" Nov 04 05:26:41.207561 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:41.207490 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9bgz9" event={"ID":"9df1fb81-e74f-45cb-b07a-026f323567b5","Type":"ContainerStarted","Data":"d175588c99b6752c8a9c4f1f873a127f46731385d8ee221718af599cfa604de2"} Nov 04 05:26:41.328823 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:41.328770 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-z2pjx" podStartSLOduration=4.058480836 podStartE2EDuration="9.328756708s" totalImagesPullingTime="5.270275872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:26:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:40.556402815 +0000 UTC m=+9.001052521" watchObservedRunningTime="2026-11-04 05:26:41.328756708 +0000 UTC m=+9.773406413" Nov 04 05:26:41.329630 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:41.329589 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9bgz9" podStartSLOduration=2.969057613 podStartE2EDuration="9.329575357s" totalImagesPullingTime="6.360517744s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:41.32857497 +0000 UTC m=+9.773224673" watchObservedRunningTime="2026-11-04 05:26:41.329575357 +0000 UTC m=+9.774225068" Nov 04 05:26:41.795438 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:41.795351 2575 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-z2pjx" Nov 04 05:26:41.796592 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:41.796568 2575 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-z2pjx" Nov 04 05:26:41.937779 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:41.937745 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs\") pod \"network-metrics-daemon-chl9q\" (UID: \"b0e60cf0-1764-4c18-ab4e-da874feaff36\") " pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:41.937942 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:41.937796 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2bc5\" (UniqueName: \"kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5\") pod \"network-check-target-cmc7w\" (UID: \"23cf3ade-4fac-44f1-813c-7f61427e2b83\") " pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:26:41.937942 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:41.937848 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:41.938064 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:41.937986 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:41.938064 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:41.938059 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret podName:5a710da8-8925-4e16-b43d-fc06a999569b nodeName:}" failed. No retries permitted until 2026-11-04 05:26:49.938026504 +0000 UTC m=+18.382676214 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret") pod "global-pull-secret-syncer-4pvss" (UID: "5a710da8-8925-4e16-b43d-fc06a999569b") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:41.938187 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:41.938126 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:41.938187 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:41.938164 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs podName:b0e60cf0-1764-4c18-ab4e-da874feaff36 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:49.938152101 +0000 UTC m=+18.382801790 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs") pod "network-metrics-daemon-chl9q" (UID: "b0e60cf0-1764-4c18-ab4e-da874feaff36") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:41.938284 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:41.938243 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:26:41.938284 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:41.938257 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:26:41.938284 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:41.938269 2575 projected.go:196] Error preparing data for projected volume kube-api-access-r2bc5 for pod openshift-network-diagnostics/network-check-target-cmc7w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:41.938429 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:41.938302 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5 podName:23cf3ade-4fac-44f1-813c-7f61427e2b83 nodeName:}" failed. No retries permitted until 2026-11-04 05:26:49.93829108 +0000 UTC m=+18.382940770 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-r2bc5" (UniqueName: "kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5") pod "network-check-target-cmc7w" (UID: "23cf3ade-4fac-44f1-813c-7f61427e2b83") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:42.127565 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:42.127209 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:42.127565 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:42.127323 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4pvss" podUID="5a710da8-8925-4e16-b43d-fc06a999569b" Nov 04 05:26:42.127565 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:42.127389 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:42.127565 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:42.127468 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-chl9q" podUID="b0e60cf0-1764-4c18-ab4e-da874feaff36" Nov 04 05:26:42.128239 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:42.127942 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:26:42.128239 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:42.128062 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmc7w" podUID="23cf3ade-4fac-44f1-813c-7f61427e2b83" Nov 04 05:26:42.213092 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:42.213059 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" event={"ID":"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050","Type":"ContainerStarted","Data":"c779cdc1ee5c665c109c31d7beb7c587cb63acbcc57b114a9f7211cb9df71516"} Nov 04 05:26:43.215196 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:43.215162 2575 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 05:26:43.660334 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:43.660173 2575 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Nov 04 05:26:44.122260 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:44.122106 2575 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T05:26:43.660330865Z","UUID":"941feb3b-4f7a-4c84-9f3b-8af82c6c5ad7","Handler":null,"Name":"","Endpoint":""} Nov 04 05:26:44.125234 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:44.125192 2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Nov 04 05:26:44.125234 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:44.125224 2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Nov 04 05:26:44.125764 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:44.125641 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:44.125851 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:44.125789 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:44.125851 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:44.125799 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4pvss" podUID="5a710da8-8925-4e16-b43d-fc06a999569b" Nov 04 05:26:44.125989 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:44.125917 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-chl9q" podUID="b0e60cf0-1764-4c18-ab4e-da874feaff36" Nov 04 05:26:44.125989 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:44.125983 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:26:44.126162 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:44.126074 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmc7w" podUID="23cf3ade-4fac-44f1-813c-7f61427e2b83" Nov 04 05:26:44.219371 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:44.219342 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" event={"ID":"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050","Type":"ContainerStarted","Data":"3b5b0e118e8b937889f824578327fd8e42e2291f8b596628d86f22651b337d5c"} Nov 04 05:26:44.220996 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:44.220966 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7b7zs" event={"ID":"bf3a5689-535b-464f-9668-8d389d6396c5","Type":"ContainerStarted","Data":"674ba02335eefbd236fb8a52a3607b2265d2218640596a31b538ad2aa6b83d16"} Nov 04 05:26:44.222953 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:44.222926 2575 generic.go:356] "Generic (PLEG): container finished" podID="99a606bd-b4a1-43e9-8525-a7afcd1eae0f" containerID="ccdd9f0a1633cc0b5fa9ef34d68e014e1a9aad72f85ed193fa028cbdae0e1dc9" exitCode=0 Nov 04 05:26:44.223073 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:44.223000 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzbgk" event={"ID":"99a606bd-b4a1-43e9-8525-a7afcd1eae0f","Type":"ContainerDied","Data":"ccdd9f0a1633cc0b5fa9ef34d68e014e1a9aad72f85ed193fa028cbdae0e1dc9"} Nov 04 05:26:44.225528 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:44.225024 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nn7ng" event={"ID":"ac2e1d23-6505-4936-9c05-70a76f8fd886","Type":"ContainerStarted","Data":"53e40e5251fcd2c36a5cbe1998e1d68fdd57e9404abd0fa41b5d89922068ec3f"} Nov 04 05:26:44.340786 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:44.340737 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-7b7zs" podStartSLOduration=3.365478366 podStartE2EDuration="12.340720737s" totalImagesPullingTime="8.975242371s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:44.340322316 +0000 UTC m=+12.784972023" watchObservedRunningTime="2026-11-04 05:26:44.340720737 +0000 UTC m=+12.785370443" Nov 04 05:26:44.463172 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:44.463067 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-nn7ng" podStartSLOduration=3.794856398 podStartE2EDuration="12.463029101s" totalImagesPullingTime="8.668172703s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:44.462401185 +0000 UTC m=+12.907050895" watchObservedRunningTime="2026-11-04 05:26:44.463029101 +0000 UTC m=+12.907678807" Nov 04 05:26:45.228491 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:45.228447 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mbh6p" event={"ID":"6c4e843c-e392-42ed-b842-508a67b0f64b","Type":"ContainerStarted","Data":"01ef85fa33b0edb828d0d10c7fb28efe7334ba0ca4b2b642d13410bd1e981065"} Nov 04 05:26:45.230356 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:45.230325 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dff2s" event={"ID":"4fc979bb-e663-4465-a2cc-16cb06be4963","Type":"ContainerStarted","Data":"aa54c903c69ff3832673746091598138c511914dee809b108c82645c35242c1c"} Nov 04 05:26:45.348609 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:45.348502 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mbh6p" podStartSLOduration=4.594835076 podStartE2EDuration="13.348483654s" totalImagesPullingTime="8.753648578s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:45.347498657 +0000 UTC m=+13.792148364" watchObservedRunningTime="2026-11-04 05:26:45.348483654 +0000 UTC m=+13.793133361" Nov 04 05:26:45.469869 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:45.469810 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dff2s" podStartSLOduration=2.960873014 podStartE2EDuration="13.469790588s" totalImagesPullingTime="10.508917574s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:45.469403663 +0000 UTC m=+13.914053371" watchObservedRunningTime="2026-11-04 05:26:45.469790588 +0000 UTC m=+13.914440298" Nov 04 05:26:46.123691 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:46.123607 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:46.123691 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:46.123648 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:46.123867 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:46.123613 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:26:46.123867 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:46.123750 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4pvss" podUID="5a710da8-8925-4e16-b43d-fc06a999569b" Nov 04 05:26:46.123867 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:46.123829 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmc7w" podUID="23cf3ade-4fac-44f1-813c-7f61427e2b83" Nov 04 05:26:46.123969 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:46.123915 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-chl9q" podUID="b0e60cf0-1764-4c18-ab4e-da874feaff36" Nov 04 05:26:46.234532 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:46.234495 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" event={"ID":"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050","Type":"ContainerStarted","Data":"81f1224400c294a6d07166171fb12adbb10999054f3b7d66e8c8e953b1386073"} Nov 04 05:26:46.236681 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:46.236591 2575 generic.go:356] "Generic (PLEG): container finished" podID="99a606bd-b4a1-43e9-8525-a7afcd1eae0f" containerID="b156e52ba7a890c7101a1737f7be98d60b976087f01c1edba666ff1eee41ba9b" exitCode=0 Nov 04 05:26:46.236681 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:46.236632 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzbgk" event={"ID":"99a606bd-b4a1-43e9-8525-a7afcd1eae0f","Type":"ContainerDied","Data":"b156e52ba7a890c7101a1737f7be98d60b976087f01c1edba666ff1eee41ba9b"} Nov 04 05:26:46.361395 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:46.361335 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" podStartSLOduration=3.625813697 podStartE2EDuration="14.361316853s" totalImagesPullingTime="10.735503156s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:46.360853584 +0000 UTC m=+14.805503293" watchObservedRunningTime="2026-11-04 05:26:46.361316853 +0000 UTC m=+14.805966560" Nov 04 05:26:48.124502 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:48.124238 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:48.124502 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:48.124315 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:26:48.124502 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:48.124492 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4pvss" podUID="5a710da8-8925-4e16-b43d-fc06a999569b" Nov 04 05:26:48.125094 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:48.124356 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:48.125094 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:48.124562 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmc7w" podUID="23cf3ade-4fac-44f1-813c-7f61427e2b83" Nov 04 05:26:48.125094 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:48.124643 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-chl9q" podUID="b0e60cf0-1764-4c18-ab4e-da874feaff36" Nov 04 05:26:48.244227 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:48.244199 2575 generic.go:356] "Generic (PLEG): container finished" podID="99a606bd-b4a1-43e9-8525-a7afcd1eae0f" containerID="32d2f9d7fe8426dc8494cc2a13944d887cd31c1189c4e7073693b07abc35e91b" exitCode=0 Nov 04 05:26:48.244375 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:48.244251 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzbgk" event={"ID":"99a606bd-b4a1-43e9-8525-a7afcd1eae0f","Type":"ContainerDied","Data":"32d2f9d7fe8426dc8494cc2a13944d887cd31c1189c4e7073693b07abc35e91b"} Nov 04 05:26:50.010465 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:50.010432 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs\") pod \"network-metrics-daemon-chl9q\" (UID: \"b0e60cf0-1764-4c18-ab4e-da874feaff36\") " pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:50.010941 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:50.010477 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2bc5\" (UniqueName: \"kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5\") pod \"network-check-target-cmc7w\" (UID: \"23cf3ade-4fac-44f1-813c-7f61427e2b83\") " pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:26:50.010941 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:50.010527 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:50.010941 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:50.010600 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:50.010941 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:50.010621 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:50.010941 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:50.010677 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret podName:5a710da8-8925-4e16-b43d-fc06a999569b nodeName:}" failed. No retries permitted until 2026-11-04 05:27:06.010657305 +0000 UTC m=+34.455307004 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret") pod "global-pull-secret-syncer-4pvss" (UID: "5a710da8-8925-4e16-b43d-fc06a999569b") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:26:50.010941 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:50.010683 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:26:50.010941 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:50.010692 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs podName:b0e60cf0-1764-4c18-ab4e-da874feaff36 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:06.010684776 +0000 UTC m=+34.455334460 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs") pod "network-metrics-daemon-chl9q" (UID: "b0e60cf0-1764-4c18-ab4e-da874feaff36") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:26:50.010941 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:50.010708 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:26:50.010941 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:50.010722 2575 projected.go:196] Error preparing data for projected volume kube-api-access-r2bc5 for pod openshift-network-diagnostics/network-check-target-cmc7w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:50.010941 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:50.010774 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5 podName:23cf3ade-4fac-44f1-813c-7f61427e2b83 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:06.010757566 +0000 UTC m=+34.455407253 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-r2bc5" (UniqueName: "kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5") pod "network-check-target-cmc7w" (UID: "23cf3ade-4fac-44f1-813c-7f61427e2b83") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:26:50.123987 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:50.123953 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:26:50.123987 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:50.123982 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:50.124242 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:50.123953 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:50.124242 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:50.124089 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmc7w" podUID="23cf3ade-4fac-44f1-813c-7f61427e2b83" Nov 04 05:26:50.124242 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:50.124186 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-chl9q" podUID="b0e60cf0-1764-4c18-ab4e-da874feaff36" Nov 04 05:26:50.124369 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:50.124273 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4pvss" podUID="5a710da8-8925-4e16-b43d-fc06a999569b" Nov 04 05:26:51.252788 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:51.252734 2575 generic.go:356] "Generic (PLEG): container finished" podID="99a606bd-b4a1-43e9-8525-a7afcd1eae0f" containerID="636274521a8199609c88ba5c555e3a63e5ae8b68a8fd8463735dc9e54e3b8671" exitCode=0 Nov 04 05:26:51.253327 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:51.252784 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzbgk" event={"ID":"99a606bd-b4a1-43e9-8525-a7afcd1eae0f","Type":"ContainerDied","Data":"636274521a8199609c88ba5c555e3a63e5ae8b68a8fd8463735dc9e54e3b8671"} Nov 04 05:26:52.124472 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:52.124439 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:52.124651 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:52.124555 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4pvss" podUID="5a710da8-8925-4e16-b43d-fc06a999569b" Nov 04 05:26:52.124929 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:52.124909 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:52.125046 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:52.125004 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-chl9q" podUID="b0e60cf0-1764-4c18-ab4e-da874feaff36" Nov 04 05:26:52.125120 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:52.125095 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:26:52.125267 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:52.125236 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmc7w" podUID="23cf3ade-4fac-44f1-813c-7f61427e2b83" Nov 04 05:26:52.259046 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:52.258998 2575 generic.go:356] "Generic (PLEG): container finished" podID="99a606bd-b4a1-43e9-8525-a7afcd1eae0f" containerID="a6a0e1cddd62321593a716f1f406b3a4569d3446ef3ff0be8b50c83c8f4b8433" exitCode=0 Nov 04 05:26:52.259449 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:52.259055 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzbgk" event={"ID":"99a606bd-b4a1-43e9-8525-a7afcd1eae0f","Type":"ContainerDied","Data":"a6a0e1cddd62321593a716f1f406b3a4569d3446ef3ff0be8b50c83c8f4b8433"} Nov 04 05:26:53.264225 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:53.264198 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bzbgk" event={"ID":"99a606bd-b4a1-43e9-8525-a7afcd1eae0f","Type":"ContainerStarted","Data":"8f26323e7e1b5f35dfef93157e4d949b4128b3b9ec6c04d48421c6e727849fe1"} Nov 04 05:26:53.390719 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:53.390569 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bzbgk" podStartSLOduration=7.138059263 podStartE2EDuration="21.390558027s" totalImagesPullingTime="14.252498764s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:53.390350769 +0000 UTC m=+21.835000475" watchObservedRunningTime="2026-11-04 05:26:53.390558027 +0000 UTC m=+21.835207733" Nov 04 05:26:54.124227 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:54.124187 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:26:54.124227 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:54.124209 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:54.124476 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:54.124283 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:54.124476 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:54.124302 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmc7w" podUID="23cf3ade-4fac-44f1-813c-7f61427e2b83" Nov 04 05:26:54.124476 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:54.124356 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4pvss" podUID="5a710da8-8925-4e16-b43d-fc06a999569b" Nov 04 05:26:54.124476 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:54.124433 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-chl9q" podUID="b0e60cf0-1764-4c18-ab4e-da874feaff36" Nov 04 05:26:54.268449 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:54.268414 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" event={"ID":"e1fc5a00-d482-4aa6-955d-ce6ff425240e","Type":"ContainerStarted","Data":"c7918e3ce16da28a2a6789a4cc1a04e8bab1d054c5be42a221a85b473f0aad98"} Nov 04 05:26:54.269077 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:54.268479 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" event={"ID":"e1fc5a00-d482-4aa6-955d-ce6ff425240e","Type":"ContainerStarted","Data":"8c3b0762158007d0206c3aa9ab22c622427ec9e49f02d68eea69dbcd07f03537"} Nov 04 05:26:54.269077 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:54.268511 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" event={"ID":"e1fc5a00-d482-4aa6-955d-ce6ff425240e","Type":"ContainerStarted","Data":"1131048eeb1c17fc26c33c8f1f78fb8346b814811972e13020a397b52dd8ea98"} Nov 04 05:26:54.269077 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:54.268539 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" event={"ID":"e1fc5a00-d482-4aa6-955d-ce6ff425240e","Type":"ContainerStarted","Data":"8495b5f7e0892fa7c55dc918b7a30f9ead28372dabde94e7713b22a40f913d47"} Nov 04 05:26:54.269077 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:54.268566 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" event={"ID":"e1fc5a00-d482-4aa6-955d-ce6ff425240e","Type":"ContainerStarted","Data":"59c69b13a24b2ff03ed2d6573e48a4e43573e6f40288c0c7a41afa08e3fb495e"} Nov 04 05:26:54.269077 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:54.268594 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" event={"ID":"e1fc5a00-d482-4aa6-955d-ce6ff425240e","Type":"ContainerStarted","Data":"4342b2291c23a04de13caf92e0c2ca685cd8eacf669b42eefc2d590bb2164637"} Nov 04 05:26:56.123717 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:56.123679 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:56.123717 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:56.123704 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:26:56.124132 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:56.123787 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4pvss" podUID="5a710da8-8925-4e16-b43d-fc06a999569b" Nov 04 05:26:56.124132 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:56.123840 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:56.124132 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:56.123915 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmc7w" podUID="23cf3ade-4fac-44f1-813c-7f61427e2b83" Nov 04 05:26:56.124132 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:56.123994 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-chl9q" podUID="b0e60cf0-1764-4c18-ab4e-da874feaff36" Nov 04 05:26:56.275209 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:56.275176 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" event={"ID":"e1fc5a00-d482-4aa6-955d-ce6ff425240e","Type":"ContainerStarted","Data":"627a8f944def7ea137f0e82e62cd87a377b8b2cc6b56a147bd8954ada2bf137f"} Nov 04 05:26:58.123461 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:58.123429 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:26:58.124139 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:58.123432 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:26:58.124139 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:58.123530 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4pvss" podUID="5a710da8-8925-4e16-b43d-fc06a999569b" Nov 04 05:26:58.124139 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:58.123436 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:26:58.124139 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:58.123624 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmc7w" podUID="23cf3ade-4fac-44f1-813c-7f61427e2b83" Nov 04 05:26:58.124139 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:26:58.123744 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-chl9q" podUID="b0e60cf0-1764-4c18-ab4e-da874feaff36" Nov 04 05:26:58.282612 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:58.282578 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" event={"ID":"e1fc5a00-d482-4aa6-955d-ce6ff425240e","Type":"ContainerStarted","Data":"b07c6335445d79a501234ac6bb707f36bcf454a8b6ca6c922752fa3b9b3fd585"} Nov 04 05:26:58.282863 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:58.282836 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:58.297531 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:58.297487 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:58.572425 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:58.572324 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" podStartSLOduration=7.822370888 podStartE2EDuration="26.572310831s" totalImagesPullingTime="18.749939943s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:26:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:26:58.519463301 +0000 UTC m=+26.964113007" watchObservedRunningTime="2026-11-04 05:26:58.572310831 +0000 UTC m=+27.016960533" Nov 04 05:26:58.975351 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:58.975315 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:59.285324 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:59.285251 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:26:59.300110 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:26:59.300082 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:00.123850 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:00.123815 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:27:00.123850 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:00.123836 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:27:00.124109 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:00.123901 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:27:00.124109 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:00.123989 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4pvss" podUID="5a710da8-8925-4e16-b43d-fc06a999569b" Nov 04 05:27:00.124109 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:00.124076 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-chl9q" podUID="b0e60cf0-1764-4c18-ab4e-da874feaff36" Nov 04 05:27:00.124265 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:00.124139 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmc7w" podUID="23cf3ade-4fac-44f1-813c-7f61427e2b83" Nov 04 05:27:00.357392 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:00.357370 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4pvss"] Nov 04 05:27:00.357724 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:00.357456 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:27:00.357724 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:00.357529 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4pvss" podUID="5a710da8-8925-4e16-b43d-fc06a999569b" Nov 04 05:27:00.360277 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:00.360258 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cmc7w"] Nov 04 05:27:00.360361 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:00.360324 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:27:00.360412 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:00.360398 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmc7w" podUID="23cf3ade-4fac-44f1-813c-7f61427e2b83" Nov 04 05:27:00.360745 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:00.360727 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-chl9q"] Nov 04 05:27:00.360829 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:00.360817 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:27:00.360917 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:00.360901 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-chl9q" podUID="b0e60cf0-1764-4c18-ab4e-da874feaff36" Nov 04 05:27:02.069818 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:02.069752 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-z2pjx" Nov 04 05:27:02.070263 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:02.069890 2575 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 05:27:02.070483 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:02.070463 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-z2pjx" Nov 04 05:27:02.126331 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:02.126305 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:27:02.126484 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:02.126372 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:27:02.126484 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:02.126402 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:27:02.126590 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:02.126526 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-chl9q" podUID="b0e60cf0-1764-4c18-ab4e-da874feaff36" Nov 04 05:27:02.126649 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:02.126586 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4pvss" podUID="5a710da8-8925-4e16-b43d-fc06a999569b" Nov 04 05:27:02.126703 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:02.126657 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmc7w" podUID="23cf3ade-4fac-44f1-813c-7f61427e2b83" Nov 04 05:27:04.123875 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:04.123840 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:27:04.123875 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:04.123873 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:27:04.124362 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:04.123841 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:27:04.124362 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:04.123944 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4pvss" podUID="5a710da8-8925-4e16-b43d-fc06a999569b" Nov 04 05:27:04.124362 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:04.124027 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmc7w" podUID="23cf3ade-4fac-44f1-813c-7f61427e2b83" Nov 04 05:27:04.124362 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:04.124218 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-chl9q" podUID="b0e60cf0-1764-4c18-ab4e-da874feaff36" Nov 04 05:27:06.024308 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.024221 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs\") pod \"network-metrics-daemon-chl9q\" (UID: \"b0e60cf0-1764-4c18-ab4e-da874feaff36\") " pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:27:06.024308 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.024260 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2bc5\" (UniqueName: \"kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5\") pod \"network-check-target-cmc7w\" (UID: \"23cf3ade-4fac-44f1-813c-7f61427e2b83\") " pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:27:06.024308 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.024286 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:27:06.024784 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:06.024393 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 05:27:06.024784 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:06.024393 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:27:06.024784 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:06.024414 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 05:27:06.024784 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:06.024435 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 05:27:06.024784 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:06.024448 2575 projected.go:196] Error preparing data for projected volume kube-api-access-r2bc5 for pod openshift-network-diagnostics/network-check-target-cmc7w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:27:06.024784 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:06.024459 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret podName:5a710da8-8925-4e16-b43d-fc06a999569b nodeName:}" failed. No retries permitted until 2026-11-04 05:27:38.024439568 +0000 UTC m=+66.469089273 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret") pod "global-pull-secret-syncer-4pvss" (UID: "5a710da8-8925-4e16-b43d-fc06a999569b") : object "kube-system"/"original-pull-secret" not registered Nov 04 05:27:06.024784 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:06.024477 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs podName:b0e60cf0-1764-4c18-ab4e-da874feaff36 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:38.024467286 +0000 UTC m=+66.469116974 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs") pod "network-metrics-daemon-chl9q" (UID: "b0e60cf0-1764-4c18-ab4e-da874feaff36") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 05:27:06.024784 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:06.024490 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5 podName:23cf3ade-4fac-44f1-813c-7f61427e2b83 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:38.024482593 +0000 UTC m=+66.469132279 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-r2bc5" (UniqueName: "kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5") pod "network-check-target-cmc7w" (UID: "23cf3ade-4fac-44f1-813c-7f61427e2b83") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 05:27:06.123564 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.123529 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:27:06.123564 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.123545 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:27:06.123780 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:06.123640 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmc7w" podUID="23cf3ade-4fac-44f1-813c-7f61427e2b83" Nov 04 05:27:06.123780 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:06.123704 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4pvss" podUID="5a710da8-8925-4e16-b43d-fc06a999569b" Nov 04 05:27:06.123889 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.123778 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:27:06.123889 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:06.123876 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-chl9q" podUID="b0e60cf0-1764-4c18-ab4e-da874feaff36" Nov 04 05:27:06.299558 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.299488 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeReady" Nov 04 05:27:06.299694 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.299632 2575 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Nov 04 05:27:06.421555 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.421524 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8fgr7"] Nov 04 05:27:06.476597 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.476570 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-g2hdc"] Nov 04 05:27:06.476738 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.476693 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:06.489885 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.489864 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g2hdc"] Nov 04 05:27:06.489982 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.489897 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8fgr7"] Nov 04 05:27:06.490021 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.489980 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g2hdc" Nov 04 05:27:06.531617 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.531594 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Nov 04 05:27:06.531702 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.531682 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Nov 04 05:27:06.531813 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.531798 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tb5s5\"" Nov 04 05:27:06.629431 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.629404 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bb1115f-702d-4b53-8852-4b38e452edb6-tmp-dir\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:06.629567 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.629438 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5bb1115f-702d-4b53-8852-4b38e452edb6-config-volume\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:06.629567 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.629461 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5bb1115f-702d-4b53-8852-4b38e452edb6-metrics-tls\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:06.629567 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.629524 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1136554-e895-435d-a306-02744ddefdb2-cert\") pod \"ingress-canary-g2hdc\" (UID: \"a1136554-e895-435d-a306-02744ddefdb2\") " pod="openshift-ingress-canary/ingress-canary-g2hdc" Nov 04 05:27:06.629661 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.629571 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmnb5\" (UniqueName: \"kubernetes.io/projected/5bb1115f-702d-4b53-8852-4b38e452edb6-kube-api-access-rmnb5\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:06.629661 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.629587 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqkfd\" (UniqueName: \"kubernetes.io/projected/a1136554-e895-435d-a306-02744ddefdb2-kube-api-access-vqkfd\") pod \"ingress-canary-g2hdc\" (UID: \"a1136554-e895-435d-a306-02744ddefdb2\") " pod="openshift-ingress-canary/ingress-canary-g2hdc" Nov 04 05:27:06.632509 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.632491 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Nov 04 05:27:06.632664 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.632649 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Nov 04 05:27:06.632706 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.632694 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-bnzsz\"" Nov 04 05:27:06.632738 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.632696 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Nov 04 05:27:06.730592 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.730563 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bb1115f-702d-4b53-8852-4b38e452edb6-tmp-dir\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:06.730592 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.730596 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5bb1115f-702d-4b53-8852-4b38e452edb6-config-volume\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:06.730761 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.730620 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5bb1115f-702d-4b53-8852-4b38e452edb6-metrics-tls\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:06.730761 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.730648 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1136554-e895-435d-a306-02744ddefdb2-cert\") pod \"ingress-canary-g2hdc\" (UID: \"a1136554-e895-435d-a306-02744ddefdb2\") " pod="openshift-ingress-canary/ingress-canary-g2hdc" Nov 04 05:27:06.730761 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.730688 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmnb5\" (UniqueName: \"kubernetes.io/projected/5bb1115f-702d-4b53-8852-4b38e452edb6-kube-api-access-rmnb5\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:06.730761 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.730717 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqkfd\" (UniqueName: \"kubernetes.io/projected/a1136554-e895-435d-a306-02744ddefdb2-kube-api-access-vqkfd\") pod \"ingress-canary-g2hdc\" (UID: \"a1136554-e895-435d-a306-02744ddefdb2\") " pod="openshift-ingress-canary/ingress-canary-g2hdc" Nov 04 05:27:06.730761 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:06.730734 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 05:27:06.730980 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:06.730783 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bb1115f-702d-4b53-8852-4b38e452edb6-metrics-tls podName:5bb1115f-702d-4b53-8852-4b38e452edb6 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:07.230768769 +0000 UTC m=+35.675418492 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5bb1115f-702d-4b53-8852-4b38e452edb6-metrics-tls") pod "dns-default-8fgr7" (UID: "5bb1115f-702d-4b53-8852-4b38e452edb6") : secret "dns-default-metrics-tls" not found Nov 04 05:27:06.731066 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.730988 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bb1115f-702d-4b53-8852-4b38e452edb6-tmp-dir\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:06.731360 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.731342 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5bb1115f-702d-4b53-8852-4b38e452edb6-config-volume\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:06.734638 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.734614 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1136554-e895-435d-a306-02744ddefdb2-cert\") pod \"ingress-canary-g2hdc\" (UID: \"a1136554-e895-435d-a306-02744ddefdb2\") " pod="openshift-ingress-canary/ingress-canary-g2hdc" Nov 04 05:27:06.795348 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.795310 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmnb5\" (UniqueName: \"kubernetes.io/projected/5bb1115f-702d-4b53-8852-4b38e452edb6-kube-api-access-rmnb5\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:06.795478 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.795457 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqkfd\" (UniqueName: \"kubernetes.io/projected/a1136554-e895-435d-a306-02744ddefdb2-kube-api-access-vqkfd\") pod \"ingress-canary-g2hdc\" (UID: \"a1136554-e895-435d-a306-02744ddefdb2\") " pod="openshift-ingress-canary/ingress-canary-g2hdc" Nov 04 05:27:06.836428 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.836397 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g2hdc" Nov 04 05:27:06.888913 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.888715 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qrxks"] Nov 04 05:27:06.917816 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.917787 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:06.947405 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:06.947379 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qrxks"] Nov 04 05:27:07.033653 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.033622 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h89x7\" (UniqueName: \"kubernetes.io/projected/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-api-access-h89x7\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:07.034130 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.033674 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9baf3dfd-f665-42ed-8cad-5f4571bf7194-crio-socket\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:07.034130 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.033739 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:07.034130 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.033820 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9baf3dfd-f665-42ed-8cad-5f4571bf7194-data-volume\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:07.034130 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.033839 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9baf3dfd-f665-42ed-8cad-5f4571bf7194-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:07.053398 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.053369 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Nov 04 05:27:07.053398 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.053393 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Nov 04 05:27:07.053561 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.053393 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Nov 04 05:27:07.053561 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.053397 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Nov 04 05:27:07.053561 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.053455 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ptjqz\"" Nov 04 05:27:07.127230 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.127206 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-tfj9k-j9jq9"] Nov 04 05:27:07.134781 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.134756 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9baf3dfd-f665-42ed-8cad-5f4571bf7194-crio-socket\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:07.134867 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.134789 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:07.134867 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.134822 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9baf3dfd-f665-42ed-8cad-5f4571bf7194-data-volume\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:07.134867 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.134851 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9baf3dfd-f665-42ed-8cad-5f4571bf7194-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:07.134962 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:07.134878 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Nov 04 05:27:07.134962 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.134890 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h89x7\" (UniqueName: \"kubernetes.io/projected/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-api-access-h89x7\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:07.134962 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:07.134939 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-rbac-proxy-cm podName:9baf3dfd-f665-42ed-8cad-5f4571bf7194 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:07.634917992 +0000 UTC m=+36.079567693 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qrxks" (UID: "9baf3dfd-f665-42ed-8cad-5f4571bf7194") : configmap "kube-rbac-proxy" not found Nov 04 05:27:07.135083 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.134991 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9baf3dfd-f665-42ed-8cad-5f4571bf7194-crio-socket\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:07.135083 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:07.134999 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 05:27:07.135083 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:07.135075 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baf3dfd-f665-42ed-8cad-5f4571bf7194-insights-runtime-extractor-tls podName:9baf3dfd-f665-42ed-8cad-5f4571bf7194 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:07.635061218 +0000 UTC m=+36.079710903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9baf3dfd-f665-42ed-8cad-5f4571bf7194-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qrxks" (UID: "9baf3dfd-f665-42ed-8cad-5f4571bf7194") : secret "insights-runtime-extractor-tls" not found Nov 04 05:27:07.135235 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.135215 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9baf3dfd-f665-42ed-8cad-5f4571bf7194-data-volume\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:07.147404 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.147358 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:27:07.184321 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.184298 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-tfj9k-j9jq9"] Nov 04 05:27:07.198503 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.198475 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h89x7\" (UniqueName: \"kubernetes.io/projected/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-api-access-h89x7\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:07.203431 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.203408 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Nov 04 05:27:07.203431 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.203421 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-z7zhw\"" Nov 04 05:27:07.236266 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.236233 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5bb1115f-702d-4b53-8852-4b38e452edb6-metrics-tls\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:07.236397 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:07.236349 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 05:27:07.236397 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:07.236395 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bb1115f-702d-4b53-8852-4b38e452edb6-metrics-tls podName:5bb1115f-702d-4b53-8852-4b38e452edb6 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:08.236384209 +0000 UTC m=+36.681033906 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5bb1115f-702d-4b53-8852-4b38e452edb6-metrics-tls") pod "dns-default-8fgr7" (UID: "5bb1115f-702d-4b53-8852-4b38e452edb6") : secret "dns-default-metrics-tls" not found Nov 04 05:27:07.257182 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.257156 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Nov 04 05:27:07.270865 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:27:07.270840 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1136554_e895_435d_a306_02744ddefdb2.slice/crio-4ef62602015af169a11d55bbd552150ac8a077a1aa4ca2fa69349c9de0faa96d WatchSource:0}: Error finding container 4ef62602015af169a11d55bbd552150ac8a077a1aa4ca2fa69349c9de0faa96d: Status 404 returned error can't find the container with id 4ef62602015af169a11d55bbd552150ac8a077a1aa4ca2fa69349c9de0faa96d Nov 04 05:27:07.290203 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.290184 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g2hdc"] Nov 04 05:27:07.303594 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.303567 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g2hdc" event={"ID":"a1136554-e895-435d-a306-02744ddefdb2","Type":"ContainerStarted","Data":"4ef62602015af169a11d55bbd552150ac8a077a1aa4ca2fa69349c9de0faa96d"} Nov 04 05:27:07.337166 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.337137 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a120e88-95d0-4176-ab05-586d4c2e7dab-trusted-ca-bundle\") pod \"periodic-gathering-tfj9k-j9jq9\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:27:07.337278 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.337185 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/4a120e88-95d0-4176-ab05-586d4c2e7dab-archives-path\") pod \"periodic-gathering-tfj9k-j9jq9\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:27:07.337330 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.337281 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bll88\" (UniqueName: \"kubernetes.io/projected/4a120e88-95d0-4176-ab05-586d4c2e7dab-kube-api-access-bll88\") pod \"periodic-gathering-tfj9k-j9jq9\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:27:07.337372 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.337334 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a120e88-95d0-4176-ab05-586d4c2e7dab-service-ca-bundle\") pod \"periodic-gathering-tfj9k-j9jq9\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:27:07.438399 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.438323 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/4a120e88-95d0-4176-ab05-586d4c2e7dab-archives-path\") pod \"periodic-gathering-tfj9k-j9jq9\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:27:07.438399 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.438375 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bll88\" (UniqueName: \"kubernetes.io/projected/4a120e88-95d0-4176-ab05-586d4c2e7dab-kube-api-access-bll88\") pod \"periodic-gathering-tfj9k-j9jq9\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:27:07.438572 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.438419 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a120e88-95d0-4176-ab05-586d4c2e7dab-service-ca-bundle\") pod \"periodic-gathering-tfj9k-j9jq9\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:27:07.438572 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.438452 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a120e88-95d0-4176-ab05-586d4c2e7dab-trusted-ca-bundle\") pod \"periodic-gathering-tfj9k-j9jq9\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:27:07.438780 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.438756 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/4a120e88-95d0-4176-ab05-586d4c2e7dab-archives-path\") pod \"periodic-gathering-tfj9k-j9jq9\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:27:07.438957 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.438940 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a120e88-95d0-4176-ab05-586d4c2e7dab-service-ca-bundle\") pod \"periodic-gathering-tfj9k-j9jq9\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:27:07.439230 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.439213 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a120e88-95d0-4176-ab05-586d4c2e7dab-trusted-ca-bundle\") pod \"periodic-gathering-tfj9k-j9jq9\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:27:07.502004 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.501975 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bll88\" (UniqueName: \"kubernetes.io/projected/4a120e88-95d0-4176-ab05-586d4c2e7dab-kube-api-access-bll88\") pod \"periodic-gathering-tfj9k-j9jq9\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:27:07.639557 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.639524 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:07.639715 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.639571 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9baf3dfd-f665-42ed-8cad-5f4571bf7194-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:07.639715 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:07.639646 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Nov 04 05:27:07.639715 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:07.639661 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 05:27:07.639812 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:07.639716 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-rbac-proxy-cm podName:9baf3dfd-f665-42ed-8cad-5f4571bf7194 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:08.639696568 +0000 UTC m=+37.084346252 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qrxks" (UID: "9baf3dfd-f665-42ed-8cad-5f4571bf7194") : configmap "kube-rbac-proxy" not found Nov 04 05:27:07.639812 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:07.639738 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baf3dfd-f665-42ed-8cad-5f4571bf7194-insights-runtime-extractor-tls podName:9baf3dfd-f665-42ed-8cad-5f4571bf7194 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:08.639727584 +0000 UTC m=+37.084377290 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9baf3dfd-f665-42ed-8cad-5f4571bf7194-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qrxks" (UID: "9baf3dfd-f665-42ed-8cad-5f4571bf7194") : secret "insights-runtime-extractor-tls" not found Nov 04 05:27:07.785131 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:07.785021 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:27:08.123808 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:08.123774 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:27:08.123808 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:08.123790 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:27:08.124478 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:08.123979 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:27:08.178884 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:27:08.178860 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a120e88_95d0_4176_ab05_586d4c2e7dab.slice/crio-fbaa1a138a1355400c58c2fbd44b7cd69611c765efe58c7e2ae239c8086b05c6 WatchSource:0}: Error finding container fbaa1a138a1355400c58c2fbd44b7cd69611c765efe58c7e2ae239c8086b05c6: Status 404 returned error can't find the container with id fbaa1a138a1355400c58c2fbd44b7cd69611c765efe58c7e2ae239c8086b05c6 Nov 04 05:27:08.180855 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:08.180677 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 05:27:08.180855 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:08.180704 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 05:27:08.180970 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:08.180950 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 05:27:08.181045 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:08.180955 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cbfrj\"" Nov 04 05:27:08.181045 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:08.180970 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 05:27:08.181045 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:08.180993 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-2psbf\"" Nov 04 05:27:08.181164 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:08.181059 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-k2gkz\"" Nov 04 05:27:08.181879 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:08.181859 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-tfj9k-j9jq9"] Nov 04 05:27:08.244933 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:08.244907 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5bb1115f-702d-4b53-8852-4b38e452edb6-metrics-tls\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:08.245075 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:08.245023 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 05:27:08.245119 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:08.245086 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bb1115f-702d-4b53-8852-4b38e452edb6-metrics-tls podName:5bb1115f-702d-4b53-8852-4b38e452edb6 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:10.245073873 +0000 UTC m=+38.689723556 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5bb1115f-702d-4b53-8852-4b38e452edb6-metrics-tls") pod "dns-default-8fgr7" (UID: "5bb1115f-702d-4b53-8852-4b38e452edb6") : secret "dns-default-metrics-tls" not found Nov 04 05:27:08.307401 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:08.307370 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" event={"ID":"4a120e88-95d0-4176-ab05-586d4c2e7dab","Type":"ContainerStarted","Data":"fbaa1a138a1355400c58c2fbd44b7cd69611c765efe58c7e2ae239c8086b05c6"} Nov 04 05:27:08.647799 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:08.647764 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9baf3dfd-f665-42ed-8cad-5f4571bf7194-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:08.647962 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:08.647837 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:08.647962 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:08.647897 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Nov 04 05:27:08.647962 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:08.647918 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 05:27:08.647962 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:08.647947 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-rbac-proxy-cm podName:9baf3dfd-f665-42ed-8cad-5f4571bf7194 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:10.647934444 +0000 UTC m=+39.092584141 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qrxks" (UID: "9baf3dfd-f665-42ed-8cad-5f4571bf7194") : configmap "kube-rbac-proxy" not found Nov 04 05:27:08.648132 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:08.647982 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baf3dfd-f665-42ed-8cad-5f4571bf7194-insights-runtime-extractor-tls podName:9baf3dfd-f665-42ed-8cad-5f4571bf7194 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:10.647967696 +0000 UTC m=+39.092617380 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9baf3dfd-f665-42ed-8cad-5f4571bf7194-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qrxks" (UID: "9baf3dfd-f665-42ed-8cad-5f4571bf7194") : secret "insights-runtime-extractor-tls" not found Nov 04 05:27:08.749938 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:08.749908 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-hgld4"] Nov 04 05:27:08.768354 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:08.768325 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-hgld4" Nov 04 05:27:08.806427 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:08.806392 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-hgld4"] Nov 04 05:27:08.839105 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:08.839078 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Nov 04 05:27:08.839105 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:08.839106 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Nov 04 05:27:08.905599 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:08.905521 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-2ltvz\"" Nov 04 05:27:08.949978 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:08.949945 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b7cc\" (UniqueName: \"kubernetes.io/projected/00c77006-a012-4f0e-a886-69ff0150de6d-kube-api-access-6b7cc\") pod \"migrator-66df5d496b-hgld4\" (UID: \"00c77006-a012-4f0e-a886-69ff0150de6d\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-hgld4" Nov 04 05:27:09.051246 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:09.051212 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b7cc\" (UniqueName: \"kubernetes.io/projected/00c77006-a012-4f0e-a886-69ff0150de6d-kube-api-access-6b7cc\") pod \"migrator-66df5d496b-hgld4\" (UID: \"00c77006-a012-4f0e-a886-69ff0150de6d\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-hgld4" Nov 04 05:27:09.114737 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:09.114705 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b7cc\" (UniqueName: \"kubernetes.io/projected/00c77006-a012-4f0e-a886-69ff0150de6d-kube-api-access-6b7cc\") pod \"migrator-66df5d496b-hgld4\" (UID: \"00c77006-a012-4f0e-a886-69ff0150de6d\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-hgld4" Nov 04 05:27:09.393523 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:09.393487 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-hgld4" Nov 04 05:27:09.779428 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:27:09.779363 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00c77006_a012_4f0e_a886_69ff0150de6d.slice/crio-c1b7a155d8e593e9cc5948587dcc48347fc8335dbd52a99eb517fbccf3cef547 WatchSource:0}: Error finding container c1b7a155d8e593e9cc5948587dcc48347fc8335dbd52a99eb517fbccf3cef547: Status 404 returned error can't find the container with id c1b7a155d8e593e9cc5948587dcc48347fc8335dbd52a99eb517fbccf3cef547 Nov 04 05:27:09.780597 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:09.780571 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-hgld4"] Nov 04 05:27:10.263113 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:10.263074 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5bb1115f-702d-4b53-8852-4b38e452edb6-metrics-tls\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:10.263278 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:10.263231 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 05:27:10.263323 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:10.263285 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bb1115f-702d-4b53-8852-4b38e452edb6-metrics-tls podName:5bb1115f-702d-4b53-8852-4b38e452edb6 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.263272268 +0000 UTC m=+42.707921952 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5bb1115f-702d-4b53-8852-4b38e452edb6-metrics-tls") pod "dns-default-8fgr7" (UID: "5bb1115f-702d-4b53-8852-4b38e452edb6") : secret "dns-default-metrics-tls" not found Nov 04 05:27:10.314780 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:10.314739 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-hgld4" event={"ID":"00c77006-a012-4f0e-a886-69ff0150de6d","Type":"ContainerStarted","Data":"c1b7a155d8e593e9cc5948587dcc48347fc8335dbd52a99eb517fbccf3cef547"} Nov 04 05:27:10.665424 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:10.665387 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9baf3dfd-f665-42ed-8cad-5f4571bf7194-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:10.665973 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:10.665477 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:10.665973 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:10.665552 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 05:27:10.665973 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:10.665627 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baf3dfd-f665-42ed-8cad-5f4571bf7194-insights-runtime-extractor-tls podName:9baf3dfd-f665-42ed-8cad-5f4571bf7194 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.665611678 +0000 UTC m=+43.110261362 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9baf3dfd-f665-42ed-8cad-5f4571bf7194-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qrxks" (UID: "9baf3dfd-f665-42ed-8cad-5f4571bf7194") : secret "insights-runtime-extractor-tls" not found Nov 04 05:27:10.665973 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:10.665562 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Nov 04 05:27:10.665973 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:10.665702 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-rbac-proxy-cm podName:9baf3dfd-f665-42ed-8cad-5f4571bf7194 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:14.665689227 +0000 UTC m=+43.110338927 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qrxks" (UID: "9baf3dfd-f665-42ed-8cad-5f4571bf7194") : configmap "kube-rbac-proxy" not found Nov 04 05:27:12.321107 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:12.320839 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g2hdc" event={"ID":"a1136554-e895-435d-a306-02744ddefdb2","Type":"ContainerStarted","Data":"0e43873ebf24330c4f40889c290cc95b0838a1c2226ef57929424d8d1cd21203"} Nov 04 05:27:12.322342 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:12.322317 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" event={"ID":"4a120e88-95d0-4176-ab05-586d4c2e7dab","Type":"ContainerStarted","Data":"74533311ad9c9e7227db1a0d702ce84f353bbeb9c512d72ee3f827413142321b"} Nov 04 05:27:12.441985 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:12.441925 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-g2hdc" podStartSLOduration=2.340866094 podStartE2EDuration="6.441912404s" totalImagesPullingTime="4.10104631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:27:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:12.441238346 +0000 UTC m=+40.885888080" watchObservedRunningTime="2026-11-04 05:27:12.441912404 +0000 UTC m=+40.886562109" Nov 04 05:27:12.562231 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:12.562176 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" podStartSLOduration=1.929466574 podStartE2EDuration="5.562158914s" totalImagesPullingTime="3.63269234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:27:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:12.561219926 +0000 UTC m=+41.005869658" watchObservedRunningTime="2026-11-04 05:27:12.562158914 +0000 UTC m=+41.006808620" Nov 04 05:27:13.326389 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:13.326348 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-hgld4" event={"ID":"00c77006-a012-4f0e-a886-69ff0150de6d","Type":"ContainerStarted","Data":"7bf036db4ee38217d0642ac1292a817813f3f62928a033cc6a6da2c37a70196f"} Nov 04 05:27:13.448578 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:13.448524 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-hgld4" podStartSLOduration=2.661594274 podStartE2EDuration="5.448508724s" totalImagesPullingTime="2.78691445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:27:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:13.447796354 +0000 UTC m=+41.892446060" watchObservedRunningTime="2026-11-04 05:27:13.448508724 +0000 UTC m=+41.893158430" Nov 04 05:27:14.296251 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:14.296215 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5bb1115f-702d-4b53-8852-4b38e452edb6-metrics-tls\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:14.296435 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:14.296345 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 05:27:14.296435 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:14.296403 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bb1115f-702d-4b53-8852-4b38e452edb6-metrics-tls podName:5bb1115f-702d-4b53-8852-4b38e452edb6 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:22.296387242 +0000 UTC m=+50.741036947 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5bb1115f-702d-4b53-8852-4b38e452edb6-metrics-tls") pod "dns-default-8fgr7" (UID: "5bb1115f-702d-4b53-8852-4b38e452edb6") : secret "dns-default-metrics-tls" not found Nov 04 05:27:14.699954 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:14.699917 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:14.700348 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:14.699971 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9baf3dfd-f665-42ed-8cad-5f4571bf7194-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:14.700348 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:14.700112 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 05:27:14.700348 ip-10-0-130-185 kubenswrapper[2575]: E1104 05:27:14.700175 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9baf3dfd-f665-42ed-8cad-5f4571bf7194-insights-runtime-extractor-tls podName:9baf3dfd-f665-42ed-8cad-5f4571bf7194 nodeName:}" failed. No retries permitted until 2026-11-04 05:27:22.700157706 +0000 UTC m=+51.144807403 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9baf3dfd-f665-42ed-8cad-5f4571bf7194-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qrxks" (UID: "9baf3dfd-f665-42ed-8cad-5f4571bf7194") : secret "insights-runtime-extractor-tls" not found Nov 04 05:27:14.700591 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:14.700572 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:15.782356 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:15.782325 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7b7zs_bf3a5689-535b-464f-9668-8d389d6396c5/dns-node-resolver/0.log" Nov 04 05:27:16.771366 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:16.771339 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9bgz9_9df1fb81-e74f-45cb-b07a-026f323567b5/node-ca/0.log" Nov 04 05:27:17.773289 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:17.773265 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g2hdc_a1136554-e895-435d-a306-02744ddefdb2/serve-healthcheck-canary/0.log" Nov 04 05:27:18.372743 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:18.372714 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-hgld4_00c77006-a012-4f0e-a886-69ff0150de6d/migrator/0.log" Nov 04 05:27:22.354740 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:22.354708 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5bb1115f-702d-4b53-8852-4b38e452edb6-metrics-tls\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:22.358373 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:22.358351 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5bb1115f-702d-4b53-8852-4b38e452edb6-metrics-tls\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:22.418202 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:22.418169 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:22.757748 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:22.757692 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9baf3dfd-f665-42ed-8cad-5f4571bf7194-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:22.759948 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:22.759921 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9baf3dfd-f665-42ed-8cad-5f4571bf7194-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:22.785598 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:22.785571 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8fgr7"] Nov 04 05:27:22.866972 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:22.866937 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:23.230783 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:27:23.230748 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9baf3dfd_f665_42ed_8cad_5f4571bf7194.slice/crio-b09efdaca27be91789cf702b91cff41442e4c9e4b102ee2c6463478c3f82e0b7 WatchSource:0}: Error finding container b09efdaca27be91789cf702b91cff41442e4c9e4b102ee2c6463478c3f82e0b7: Status 404 returned error can't find the container with id b09efdaca27be91789cf702b91cff41442e4c9e4b102ee2c6463478c3f82e0b7 Nov 04 05:27:23.232797 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:23.232771 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qrxks"] Nov 04 05:27:23.350865 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:23.350828 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qrxks" event={"ID":"9baf3dfd-f665-42ed-8cad-5f4571bf7194","Type":"ContainerStarted","Data":"6ac44127725555860440bc210d54b2233d20b33d742d7c703de2d93b7e633e9d"} Nov 04 05:27:23.351074 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:23.350883 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qrxks" event={"ID":"9baf3dfd-f665-42ed-8cad-5f4571bf7194","Type":"ContainerStarted","Data":"b09efdaca27be91789cf702b91cff41442e4c9e4b102ee2c6463478c3f82e0b7"} Nov 04 05:27:23.351956 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:23.351933 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8fgr7" event={"ID":"5bb1115f-702d-4b53-8852-4b38e452edb6","Type":"ContainerStarted","Data":"a2cc34cd793b26d5679286cb22c1afd256a95096c2c83af5b04a0dc51c24bdcf"} Nov 04 05:27:25.358674 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:25.358638 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qrxks" event={"ID":"9baf3dfd-f665-42ed-8cad-5f4571bf7194","Type":"ContainerStarted","Data":"d452ef129961fc8ff80b86c67f170e81d34eafc36b2605ccc691aa61f164bde6"} Nov 04 05:27:25.360364 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:25.360341 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8fgr7" event={"ID":"5bb1115f-702d-4b53-8852-4b38e452edb6","Type":"ContainerStarted","Data":"3323544c09123b1c2231af3c9cf4835ac018673c474af812a465400aef19a3f4"} Nov 04 05:27:25.360483 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:25.360383 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8fgr7" event={"ID":"5bb1115f-702d-4b53-8852-4b38e452edb6","Type":"ContainerStarted","Data":"2295cbc0ae8a652c412fc0b8fca1c13d54db8ffaaa497c479a745481cdf68848"} Nov 04 05:27:25.360607 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:25.360587 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:25.530849 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:25.530794 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-8fgr7" podStartSLOduration=17.262765457 podStartE2EDuration="19.530781472s" totalImagesPullingTime="2.268016015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:27:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:25.530571215 +0000 UTC m=+53.975220969" watchObservedRunningTime="2026-11-04 05:27:25.530781472 +0000 UTC m=+53.975431192" Nov 04 05:27:28.371600 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:28.371567 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qrxks" event={"ID":"9baf3dfd-f665-42ed-8cad-5f4571bf7194","Type":"ContainerStarted","Data":"d9ff355a3b71bce49400510c79f5f17255e6a8822c53cc580d4cf8f5733ef40e"} Nov 04 05:27:28.495750 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:28.495705 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qrxks" podStartSLOduration=18.110315809 podStartE2EDuration="22.495690948s" totalImagesPullingTime="4.385375139s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:27:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:28.495489654 +0000 UTC m=+56.940139388" watchObservedRunningTime="2026-11-04 05:27:28.495690948 +0000 UTC m=+56.940340653" Nov 04 05:27:31.301285 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:31.301255 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:31.993386 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:31.993352 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pnm9q"] Nov 04 05:27:31.995425 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:31.995395 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pnm9q" Nov 04 05:27:32.031019 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.030990 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d7741ae3-0846-4449-a738-45251ad287df-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-pnm9q\" (UID: \"d7741ae3-0846-4449-a738-45251ad287df\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pnm9q" Nov 04 05:27:32.072007 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.071979 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-gd2dx\"" Nov 04 05:27:32.072190 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.072100 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Nov 04 05:27:32.083513 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.083482 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pnm9q"] Nov 04 05:27:32.112540 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.112509 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-f59mk"] Nov 04 05:27:32.114533 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.114515 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-f59mk" Nov 04 05:27:32.122050 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.122019 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74f7d9b7f7-488q9"] Nov 04 05:27:32.124306 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.124286 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.131376 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.131350 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d7741ae3-0846-4449-a738-45251ad287df-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-pnm9q\" (UID: \"d7741ae3-0846-4449-a738-45251ad287df\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pnm9q" Nov 04 05:27:32.133959 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.133934 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d7741ae3-0846-4449-a738-45251ad287df-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-pnm9q\" (UID: \"d7741ae3-0846-4449-a738-45251ad287df\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pnm9q" Nov 04 05:27:32.181813 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.181785 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-f59mk"] Nov 04 05:27:32.189997 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.189972 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Nov 04 05:27:32.189997 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.189986 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Nov 04 05:27:32.194883 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.194863 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74f7d9b7f7-488q9"] Nov 04 05:27:32.231644 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.231614 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-registry-certificates\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.231644 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.231647 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82e24a0f-8277-484a-b7b4-50dcfb2b4fdd-tmp\") pod \"downloads-6dd789c8b7-f59mk\" (UID: \"82e24a0f-8277-484a-b7b4-50dcfb2b4fdd\") " pod="openshift-console/downloads-6dd789c8b7-f59mk" Nov 04 05:27:32.231814 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.231679 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-bound-sa-token\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.231814 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.231701 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-registry-tls\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.231892 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.231867 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-ca-trust-extracted\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.231939 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.231903 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-installation-pull-secrets\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.232025 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.232005 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-image-registry-private-configuration\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.232100 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.232067 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-trusted-ca\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.232100 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.232089 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tcm2\" (UniqueName: \"kubernetes.io/projected/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-kube-api-access-2tcm2\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.232160 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.232145 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvp55\" (UniqueName: \"kubernetes.io/projected/82e24a0f-8277-484a-b7b4-50dcfb2b4fdd-kube-api-access-zvp55\") pod \"downloads-6dd789c8b7-f59mk\" (UID: \"82e24a0f-8277-484a-b7b4-50dcfb2b4fdd\") " pod="openshift-console/downloads-6dd789c8b7-f59mk" Nov 04 05:27:32.246710 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.246668 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Nov 04 05:27:32.280229 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.280197 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Nov 04 05:27:32.280455 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.280439 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-svfgt\"" Nov 04 05:27:32.280539 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.280525 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6qrg6\"" Nov 04 05:27:32.280627 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.280616 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Nov 04 05:27:32.280687 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.280678 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Nov 04 05:27:32.305331 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.305306 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pnm9q" Nov 04 05:27:32.332464 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.332441 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-bound-sa-token\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.332464 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.332471 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-registry-tls\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.332643 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.332500 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-ca-trust-extracted\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.332643 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.332516 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-installation-pull-secrets\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.332643 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.332548 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-image-registry-private-configuration\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.332643 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.332570 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-trusted-ca\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.332643 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.332586 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tcm2\" (UniqueName: \"kubernetes.io/projected/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-kube-api-access-2tcm2\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.332643 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.332608 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvp55\" (UniqueName: \"kubernetes.io/projected/82e24a0f-8277-484a-b7b4-50dcfb2b4fdd-kube-api-access-zvp55\") pod \"downloads-6dd789c8b7-f59mk\" (UID: \"82e24a0f-8277-484a-b7b4-50dcfb2b4fdd\") " pod="openshift-console/downloads-6dd789c8b7-f59mk" Nov 04 05:27:32.332991 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.332662 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-registry-certificates\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.332991 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.332690 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82e24a0f-8277-484a-b7b4-50dcfb2b4fdd-tmp\") pod \"downloads-6dd789c8b7-f59mk\" (UID: \"82e24a0f-8277-484a-b7b4-50dcfb2b4fdd\") " pod="openshift-console/downloads-6dd789c8b7-f59mk" Nov 04 05:27:32.332991 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.332958 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-ca-trust-extracted\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.333147 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.333065 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82e24a0f-8277-484a-b7b4-50dcfb2b4fdd-tmp\") pod \"downloads-6dd789c8b7-f59mk\" (UID: \"82e24a0f-8277-484a-b7b4-50dcfb2b4fdd\") " pod="openshift-console/downloads-6dd789c8b7-f59mk" Nov 04 05:27:32.334082 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.334057 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-trusted-ca\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.334082 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.334072 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-registry-certificates\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.335182 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.335161 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-registry-tls\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.335287 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.335190 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-installation-pull-secrets\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.335343 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.335281 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-image-registry-private-configuration\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.398247 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.398217 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-bound-sa-token\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.398486 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.398461 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvp55\" (UniqueName: \"kubernetes.io/projected/82e24a0f-8277-484a-b7b4-50dcfb2b4fdd-kube-api-access-zvp55\") pod \"downloads-6dd789c8b7-f59mk\" (UID: \"82e24a0f-8277-484a-b7b4-50dcfb2b4fdd\") " pod="openshift-console/downloads-6dd789c8b7-f59mk" Nov 04 05:27:32.398531 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.398485 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tcm2\" (UniqueName: \"kubernetes.io/projected/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-kube-api-access-2tcm2\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.426418 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.426394 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-f59mk" Nov 04 05:27:32.434931 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.434908 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:32.667740 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:27:32.667715 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7741ae3_0846_4449_a738_45251ad287df.slice/crio-0a960f30f55e9ac08398126969050a8c92023b2d69187c0d0eea1a543fea9c06 WatchSource:0}: Error finding container 0a960f30f55e9ac08398126969050a8c92023b2d69187c0d0eea1a543fea9c06: Status 404 returned error can't find the container with id 0a960f30f55e9ac08398126969050a8c92023b2d69187c0d0eea1a543fea9c06 Nov 04 05:27:32.670522 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.670501 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pnm9q"] Nov 04 05:27:32.792900 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:27:32.792871 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82e24a0f_8277_484a_b7b4_50dcfb2b4fdd.slice/crio-b6c2dbf7b1476ea572b7848a417ec1412e9c6f2704f5e77696912d547c9e8c3e WatchSource:0}: Error finding container b6c2dbf7b1476ea572b7848a417ec1412e9c6f2704f5e77696912d547c9e8c3e: Status 404 returned error can't find the container with id b6c2dbf7b1476ea572b7848a417ec1412e9c6f2704f5e77696912d547c9e8c3e Nov 04 05:27:32.795231 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.795211 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-f59mk"] Nov 04 05:27:32.841000 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:27:32.840964 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4806f73f_d8d3_41a3_865f_9df7fe88e1b8.slice/crio-f24431ce12e2fb36139b8e783f12f8402f9c33ee32afe71c6d090c111a9bb5b2 WatchSource:0}: Error finding container f24431ce12e2fb36139b8e783f12f8402f9c33ee32afe71c6d090c111a9bb5b2: Status 404 returned error can't find the container with id f24431ce12e2fb36139b8e783f12f8402f9c33ee32afe71c6d090c111a9bb5b2 Nov 04 05:27:32.846563 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:32.844692 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74f7d9b7f7-488q9"] Nov 04 05:27:33.387400 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:33.387357 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-f59mk" event={"ID":"82e24a0f-8277-484a-b7b4-50dcfb2b4fdd","Type":"ContainerStarted","Data":"b6c2dbf7b1476ea572b7848a417ec1412e9c6f2704f5e77696912d547c9e8c3e"} Nov 04 05:27:33.388821 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:33.388795 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" event={"ID":"4806f73f-d8d3-41a3-865f-9df7fe88e1b8","Type":"ContainerStarted","Data":"369ffb4259f4a6fe7237954c925dc6ceaf4e4d85fa22a702612d3877246c94d4"} Nov 04 05:27:33.388941 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:33.388853 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" event={"ID":"4806f73f-d8d3-41a3-865f-9df7fe88e1b8","Type":"ContainerStarted","Data":"f24431ce12e2fb36139b8e783f12f8402f9c33ee32afe71c6d090c111a9bb5b2"} Nov 04 05:27:33.389058 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:33.389020 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:33.389952 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:33.389934 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pnm9q" event={"ID":"d7741ae3-0846-4449-a738-45251ad287df","Type":"ContainerStarted","Data":"0a960f30f55e9ac08398126969050a8c92023b2d69187c0d0eea1a543fea9c06"} Nov 04 05:27:33.514531 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:33.514484 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" podStartSLOduration=1.5144701619999998 podStartE2EDuration="1.514470162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:27:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:33.51348277 +0000 UTC m=+61.958132478" watchObservedRunningTime="2026-11-04 05:27:33.514470162 +0000 UTC m=+61.959119867" Nov 04 05:27:35.365912 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:35.365885 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:35.399533 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:35.399497 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pnm9q" event={"ID":"d7741ae3-0846-4449-a738-45251ad287df","Type":"ContainerStarted","Data":"8ce774b767550240543fb798c8d6258582ab9fd92fa13cac7c97c5e366735eb2"} Nov 04 05:27:35.399747 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:35.399723 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pnm9q" Nov 04 05:27:35.405239 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:35.405216 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pnm9q" Nov 04 05:27:35.610081 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:35.610018 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pnm9q" podStartSLOduration=2.8600166 podStartE2EDuration="4.610002337s" totalImagesPullingTime="1.749985737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:27:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:35.609120328 +0000 UTC m=+64.053770033" watchObservedRunningTime="2026-11-04 05:27:35.610002337 +0000 UTC m=+64.054652043" Nov 04 05:27:38.080423 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:38.080382 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs\") pod \"network-metrics-daemon-chl9q\" (UID: \"b0e60cf0-1764-4c18-ab4e-da874feaff36\") " pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:27:38.080866 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:38.080426 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2bc5\" (UniqueName: \"kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5\") pod \"network-check-target-cmc7w\" (UID: \"23cf3ade-4fac-44f1-813c-7f61427e2b83\") " pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:27:38.080866 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:38.080489 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:27:38.134641 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:38.134613 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 05:27:38.187170 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:38.187133 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 05:27:38.187170 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:38.187151 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 05:27:38.193810 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:38.193785 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:27:38.193931 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:38.193810 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs\") pod \"network-metrics-daemon-chl9q\" (UID: \"b0e60cf0-1764-4c18-ab4e-da874feaff36\") " pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:27:38.194659 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:38.194641 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 05:27:38.204089 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:38.204066 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2bc5\" (UniqueName: \"kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5\") pod \"network-check-target-cmc7w\" (UID: \"23cf3ade-4fac-44f1-813c-7f61427e2b83\") " pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:27:38.257071 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:38.257025 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-2psbf\"" Nov 04 05:27:38.264100 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:38.264076 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:27:38.520603 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:38.520572 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cbfrj\"" Nov 04 05:27:38.526742 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:38.526717 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:27:38.535246 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:38.535220 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-k2gkz\"" Nov 04 05:27:38.541942 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:38.541920 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:27:38.645216 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:27:38.645189 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a710da8_8925_4e16_b43d_fc06a999569b.slice/crio-09f4f8137bae50dd458fac2cf6749649224d80f65839a31044896d947a39ed15 WatchSource:0}: Error finding container 09f4f8137bae50dd458fac2cf6749649224d80f65839a31044896d947a39ed15: Status 404 returned error can't find the container with id 09f4f8137bae50dd458fac2cf6749649224d80f65839a31044896d947a39ed15 Nov 04 05:27:38.647477 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:38.647456 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4pvss"] Nov 04 05:27:38.898568 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:27:38.898538 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0e60cf0_1764_4c18_ab4e_da874feaff36.slice/crio-a0e86e8bd29d3b22725f3b36e1864b50e3cd003841456838da9511ec22ba3da5 WatchSource:0}: Error finding container a0e86e8bd29d3b22725f3b36e1864b50e3cd003841456838da9511ec22ba3da5: Status 404 returned error can't find the container with id a0e86e8bd29d3b22725f3b36e1864b50e3cd003841456838da9511ec22ba3da5 Nov 04 05:27:38.901225 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:38.901201 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-chl9q"] Nov 04 05:27:38.943680 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:27:38.943650 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23cf3ade_4fac_44f1_813c_7f61427e2b83.slice/crio-1fa2fef846d263f0e570b3c8a9f936acd1a5f014e1a7f3ada3b50c735aa4d8f5 WatchSource:0}: Error finding container 1fa2fef846d263f0e570b3c8a9f936acd1a5f014e1a7f3ada3b50c735aa4d8f5: Status 404 returned error can't find the container with id 1fa2fef846d263f0e570b3c8a9f936acd1a5f014e1a7f3ada3b50c735aa4d8f5 Nov 04 05:27:38.946282 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:38.946260 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cmc7w"] Nov 04 05:27:39.413427 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:39.413378 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cmc7w" event={"ID":"23cf3ade-4fac-44f1-813c-7f61427e2b83","Type":"ContainerStarted","Data":"1fa2fef846d263f0e570b3c8a9f936acd1a5f014e1a7f3ada3b50c735aa4d8f5"} Nov 04 05:27:39.414643 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:39.414610 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-chl9q" event={"ID":"b0e60cf0-1764-4c18-ab4e-da874feaff36","Type":"ContainerStarted","Data":"a0e86e8bd29d3b22725f3b36e1864b50e3cd003841456838da9511ec22ba3da5"} Nov 04 05:27:39.415956 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:39.415916 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4pvss" event={"ID":"5a710da8-8925-4e16-b43d-fc06a999569b","Type":"ContainerStarted","Data":"09f4f8137bae50dd458fac2cf6749649224d80f65839a31044896d947a39ed15"} Nov 04 05:27:42.434864 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:42.434818 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-chl9q" event={"ID":"b0e60cf0-1764-4c18-ab4e-da874feaff36","Type":"ContainerStarted","Data":"be1643a9027d69c3878f3a53e67bf6b800f4e00210d99c3f1435c87e05438b7f"} Nov 04 05:27:42.435364 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:42.434909 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-chl9q" event={"ID":"b0e60cf0-1764-4c18-ab4e-da874feaff36","Type":"ContainerStarted","Data":"5da4f257fbbe95ceff413bd6906efaf1784f95eb933b9ca489c76c4ac034721a"} Nov 04 05:27:42.609972 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:42.608619 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-chl9q" podStartSLOduration=68.117658945 podStartE2EDuration="1m10.608600413s" totalImagesPullingTime="2.490941468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:26:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:42.556212538 +0000 UTC m=+71.000862262" watchObservedRunningTime="2026-11-04 05:27:42.608600413 +0000 UTC m=+71.053250120" Nov 04 05:27:43.492301 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.492271 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj"] Nov 04 05:27:43.494554 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.494531 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" Nov 04 05:27:43.555058 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.553071 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Nov 04 05:27:43.555058 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.553199 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Nov 04 05:27:43.605868 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.605833 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Nov 04 05:27:43.606067 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.605992 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-tnkgv\"" Nov 04 05:27:43.606150 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.606110 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Nov 04 05:27:43.606193 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.606162 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Nov 04 05:27:43.607087 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.607064 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj"] Nov 04 05:27:43.608172 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.608152 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj"] Nov 04 05:27:43.612658 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.612632 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:43.622002 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.621977 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4dcc3e21-060d-4d88-a034-025ee61af3ff-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-pbhxj\" (UID: \"4dcc3e21-060d-4d88-a034-025ee61af3ff\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" Nov 04 05:27:43.622133 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.622013 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4dcc3e21-060d-4d88-a034-025ee61af3ff-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-pbhxj\" (UID: \"4dcc3e21-060d-4d88-a034-025ee61af3ff\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" Nov 04 05:27:43.622449 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.622165 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4dcc3e21-060d-4d88-a034-025ee61af3ff-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-pbhxj\" (UID: \"4dcc3e21-060d-4d88-a034-025ee61af3ff\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" Nov 04 05:27:43.622449 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.622209 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvqg7\" (UniqueName: \"kubernetes.io/projected/4dcc3e21-060d-4d88-a034-025ee61af3ff-kube-api-access-tvqg7\") pod \"openshift-state-metrics-7bd7489957-pbhxj\" (UID: \"4dcc3e21-060d-4d88-a034-025ee61af3ff\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" Nov 04 05:27:43.661264 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.661191 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-69gbq"] Nov 04 05:27:43.665559 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.665512 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj"] Nov 04 05:27:43.665678 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.665619 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.668958 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.668937 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-94czp\"" Nov 04 05:27:43.668958 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.668955 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Nov 04 05:27:43.669164 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.669088 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Nov 04 05:27:43.669217 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.669182 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Nov 04 05:27:43.723388 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.723348 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7810845b-042a-4fe6-bc58-cf7285f7e396-root\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.723531 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.723395 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jpd5\" (UniqueName: \"kubernetes.io/projected/7810845b-042a-4fe6-bc58-cf7285f7e396-kube-api-access-5jpd5\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.723531 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.723436 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq2fx\" (UniqueName: \"kubernetes.io/projected/9909a4b9-77f0-4be2-9374-26a602a72987-kube-api-access-cq2fx\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:43.723531 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.723504 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-tls\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.723696 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.723557 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9909a4b9-77f0-4be2-9374-26a602a72987-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:43.723696 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.723586 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7810845b-042a-4fe6-bc58-cf7285f7e396-sys\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.723696 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.723615 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7810845b-042a-4fe6-bc58-cf7285f7e396-metrics-client-ca\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.723696 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.723662 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4dcc3e21-060d-4d88-a034-025ee61af3ff-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-pbhxj\" (UID: \"4dcc3e21-060d-4d88-a034-025ee61af3ff\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" Nov 04 05:27:43.723696 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.723688 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4dcc3e21-060d-4d88-a034-025ee61af3ff-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-pbhxj\" (UID: \"4dcc3e21-060d-4d88-a034-025ee61af3ff\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" Nov 04 05:27:43.723940 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.723714 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-textfile\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.723940 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.723757 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.723940 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.723778 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-wtmp\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.723940 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.723795 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9909a4b9-77f0-4be2-9374-26a602a72987-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:43.723940 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.723823 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9909a4b9-77f0-4be2-9374-26a602a72987-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:43.723940 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.723854 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4dcc3e21-060d-4d88-a034-025ee61af3ff-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-pbhxj\" (UID: \"4dcc3e21-060d-4d88-a034-025ee61af3ff\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" Nov 04 05:27:43.723940 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.723884 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9909a4b9-77f0-4be2-9374-26a602a72987-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:43.723940 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.723915 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-accelerators-collector-config\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.724250 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.723946 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9909a4b9-77f0-4be2-9374-26a602a72987-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:43.724250 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.723977 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvqg7\" (UniqueName: \"kubernetes.io/projected/4dcc3e21-060d-4d88-a034-025ee61af3ff-kube-api-access-tvqg7\") pod \"openshift-state-metrics-7bd7489957-pbhxj\" (UID: \"4dcc3e21-060d-4d88-a034-025ee61af3ff\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" Nov 04 05:27:43.725375 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.725302 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4dcc3e21-060d-4d88-a034-025ee61af3ff-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-pbhxj\" (UID: \"4dcc3e21-060d-4d88-a034-025ee61af3ff\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" Nov 04 05:27:43.726807 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.726736 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Nov 04 05:27:43.727004 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.726974 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-szc85\"" Nov 04 05:27:43.727119 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.727104 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Nov 04 05:27:43.727714 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.727688 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4dcc3e21-060d-4d88-a034-025ee61af3ff-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-pbhxj\" (UID: \"4dcc3e21-060d-4d88-a034-025ee61af3ff\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" Nov 04 05:27:43.728987 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.728962 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4dcc3e21-060d-4d88-a034-025ee61af3ff-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-pbhxj\" (UID: \"4dcc3e21-060d-4d88-a034-025ee61af3ff\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" Nov 04 05:27:43.760353 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.760321 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Nov 04 05:27:43.789201 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.789168 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvqg7\" (UniqueName: \"kubernetes.io/projected/4dcc3e21-060d-4d88-a034-025ee61af3ff-kube-api-access-tvqg7\") pod \"openshift-state-metrics-7bd7489957-pbhxj\" (UID: \"4dcc3e21-060d-4d88-a034-025ee61af3ff\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" Nov 04 05:27:43.810570 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.810540 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" Nov 04 05:27:43.824929 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.824899 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9909a4b9-77f0-4be2-9374-26a602a72987-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:43.825074 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.824947 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-accelerators-collector-config\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.825344 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.825325 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9909a4b9-77f0-4be2-9374-26a602a72987-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:43.826327 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.825451 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9909a4b9-77f0-4be2-9374-26a602a72987-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:43.826327 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.825493 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7810845b-042a-4fe6-bc58-cf7285f7e396-root\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.826327 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.825516 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jpd5\" (UniqueName: \"kubernetes.io/projected/7810845b-042a-4fe6-bc58-cf7285f7e396-kube-api-access-5jpd5\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.826327 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.825566 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq2fx\" (UniqueName: \"kubernetes.io/projected/9909a4b9-77f0-4be2-9374-26a602a72987-kube-api-access-cq2fx\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:43.826327 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.825590 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-tls\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.826327 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.825609 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7810845b-042a-4fe6-bc58-cf7285f7e396-root\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.826327 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.825642 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9909a4b9-77f0-4be2-9374-26a602a72987-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:43.826327 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.825665 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7810845b-042a-4fe6-bc58-cf7285f7e396-sys\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.826327 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.825692 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7810845b-042a-4fe6-bc58-cf7285f7e396-metrics-client-ca\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.826327 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.825738 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-textfile\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.826327 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.825787 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.826327 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.825820 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-wtmp\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.826327 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.825850 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9909a4b9-77f0-4be2-9374-26a602a72987-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:43.826327 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.825888 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9909a4b9-77f0-4be2-9374-26a602a72987-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:43.826327 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.826226 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9909a4b9-77f0-4be2-9374-26a602a72987-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:43.826327 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.825563 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-accelerators-collector-config\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.827187 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.826557 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7810845b-042a-4fe6-bc58-cf7285f7e396-metrics-client-ca\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.827187 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.826626 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7810845b-042a-4fe6-bc58-cf7285f7e396-sys\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.827187 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.826764 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-wtmp\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.827187 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.827144 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-textfile\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.827376 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.827361 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9909a4b9-77f0-4be2-9374-26a602a72987-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:43.828973 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.828941 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.830266 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.830244 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9909a4b9-77f0-4be2-9374-26a602a72987-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:43.830492 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.830467 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9909a4b9-77f0-4be2-9374-26a602a72987-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:43.831028 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.830995 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-tls\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.895508 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.895473 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jpd5\" (UniqueName: \"kubernetes.io/projected/7810845b-042a-4fe6-bc58-cf7285f7e396-kube-api-access-5jpd5\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:43.897469 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.897425 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq2fx\" (UniqueName: \"kubernetes.io/projected/9909a4b9-77f0-4be2-9374-26a602a72987-kube-api-access-cq2fx\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:43.933467 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.933380 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:43.985198 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:43.985170 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:44.019110 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:27:44.019075 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7810845b_042a_4fe6_bc58_cf7285f7e396.slice/crio-d353d62f4bcc28a17a12b87e478a93b66ce4b99738e4bd8066eef5b9ff8ddfc2 WatchSource:0}: Error finding container d353d62f4bcc28a17a12b87e478a93b66ce4b99738e4bd8066eef5b9ff8ddfc2: Status 404 returned error can't find the container with id d353d62f4bcc28a17a12b87e478a93b66ce4b99738e4bd8066eef5b9ff8ddfc2 Nov 04 05:27:44.242911 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:27:44.242877 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dcc3e21_060d_4d88_a034_025ee61af3ff.slice/crio-46d6e055c92ced70e25d887a457d36afe3656f32caae299a832604f3d72f46cb WatchSource:0}: Error finding container 46d6e055c92ced70e25d887a457d36afe3656f32caae299a832604f3d72f46cb: Status 404 returned error can't find the container with id 46d6e055c92ced70e25d887a457d36afe3656f32caae299a832604f3d72f46cb Nov 04 05:27:44.245224 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.245140 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj"] Nov 04 05:27:44.311635 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.311605 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-788f8bd759-f6zjh"] Nov 04 05:27:44.322537 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.322512 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.349805 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.349728 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj"] Nov 04 05:27:44.359135 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:27:44.358197 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9909a4b9_77f0_4be2_9374_26a602a72987.slice/crio-44e9081016a2d01003ff747c050f2c3c9f81da9cc01e11bbf8f204409360c56d WatchSource:0}: Error finding container 44e9081016a2d01003ff747c050f2c3c9f81da9cc01e11bbf8f204409360c56d: Status 404 returned error can't find the container with id 44e9081016a2d01003ff747c050f2c3c9f81da9cc01e11bbf8f204409360c56d Nov 04 05:27:44.367965 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.367940 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-788f8bd759-f6zjh"] Nov 04 05:27:44.383869 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.383840 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Nov 04 05:27:44.383869 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.383840 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Nov 04 05:27:44.384025 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.383841 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Nov 04 05:27:44.431301 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.431228 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11b27b07-7ada-441c-8799-e3d3d111bc18-console-oauth-config\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.431301 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.431270 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-trusted-ca-bundle\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.431474 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.431372 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11b27b07-7ada-441c-8799-e3d3d111bc18-tmp\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.431474 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.431419 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-oauth-serving-cert\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.431583 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.431495 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-console-config\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.431583 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.431545 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-service-ca\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.431678 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.431580 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11b27b07-7ada-441c-8799-e3d3d111bc18-console-serving-cert\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.431678 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.431606 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s58tj\" (UniqueName: \"kubernetes.io/projected/11b27b07-7ada-441c-8799-e3d3d111bc18-kube-api-access-s58tj\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.436830 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.436798 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Nov 04 05:27:44.436926 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.436838 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Nov 04 05:27:44.436977 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.436940 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Nov 04 05:27:44.437061 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.437024 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-sxw5c\"" Nov 04 05:27:44.444981 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.444956 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cmc7w" event={"ID":"23cf3ade-4fac-44f1-813c-7f61427e2b83","Type":"ContainerStarted","Data":"136db68c27cf8808968a994d6be392cfe0231de0a80c94fe7e7dbf47c729bb20"} Nov 04 05:27:44.445448 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.445428 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:27:44.448379 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.448347 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-69gbq" event={"ID":"7810845b-042a-4fe6-bc58-cf7285f7e396","Type":"ContainerStarted","Data":"d353d62f4bcc28a17a12b87e478a93b66ce4b99738e4bd8066eef5b9ff8ddfc2"} Nov 04 05:27:44.449924 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.449905 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" event={"ID":"9909a4b9-77f0-4be2-9374-26a602a72987","Type":"ContainerStarted","Data":"44e9081016a2d01003ff747c050f2c3c9f81da9cc01e11bbf8f204409360c56d"} Nov 04 05:27:44.452679 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.452652 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" event={"ID":"4dcc3e21-060d-4d88-a034-025ee61af3ff","Type":"ContainerStarted","Data":"ea9599179d6abc4c39f5f9358b7d5d8dfc8e610853a99999ac61d922fb7dfccf"} Nov 04 05:27:44.452775 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.452706 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" event={"ID":"4dcc3e21-060d-4d88-a034-025ee61af3ff","Type":"ContainerStarted","Data":"549a5562a81fd4acd691ab3212522a75a3be09eebb1a862e41a05dcfb0ed9672"} Nov 04 05:27:44.452775 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.452735 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" event={"ID":"4dcc3e21-060d-4d88-a034-025ee61af3ff","Type":"ContainerStarted","Data":"46d6e055c92ced70e25d887a457d36afe3656f32caae299a832604f3d72f46cb"} Nov 04 05:27:44.532339 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.532261 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11b27b07-7ada-441c-8799-e3d3d111bc18-console-oauth-config\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.532339 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.532308 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-trusted-ca-bundle\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.532868 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.532357 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11b27b07-7ada-441c-8799-e3d3d111bc18-tmp\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.532868 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.532401 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-oauth-serving-cert\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.532868 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.532431 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-console-config\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.532868 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.532461 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-service-ca\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.532868 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.532483 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11b27b07-7ada-441c-8799-e3d3d111bc18-console-serving-cert\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.532868 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.532505 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s58tj\" (UniqueName: \"kubernetes.io/projected/11b27b07-7ada-441c-8799-e3d3d111bc18-kube-api-access-s58tj\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.532868 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.532839 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11b27b07-7ada-441c-8799-e3d3d111bc18-tmp\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.533319 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.533290 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-service-ca\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.533428 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.533398 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-oauth-serving-cert\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.533496 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.533428 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-trusted-ca-bundle\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.533496 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.533476 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-console-config\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.535539 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.535515 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11b27b07-7ada-441c-8799-e3d3d111bc18-console-oauth-config\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.536422 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.536398 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11b27b07-7ada-441c-8799-e3d3d111bc18-console-serving-cert\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.571869 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.571813 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-cmc7w" podStartSLOduration=68.137870194 podStartE2EDuration="1m12.571797192s" totalImagesPullingTime="4.433926998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:26:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:44.571234927 +0000 UTC m=+73.015884652" watchObservedRunningTime="2026-11-04 05:27:44.571797192 +0000 UTC m=+73.016446901" Nov 04 05:27:44.600266 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.600239 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s58tj\" (UniqueName: \"kubernetes.io/projected/11b27b07-7ada-441c-8799-e3d3d111bc18-kube-api-access-s58tj\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:44.655462 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:44.655429 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:45.051174 ip-10-0-130-185 kubenswrapper[2575]: W1104 05:27:45.051133 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11b27b07_7ada_441c_8799_e3d3d111bc18.slice/crio-7af0d5e931332006d648b413e3d1c89f456a6b0dae1ab2e542ebad9416d483e4 WatchSource:0}: Error finding container 7af0d5e931332006d648b413e3d1c89f456a6b0dae1ab2e542ebad9416d483e4: Status 404 returned error can't find the container with id 7af0d5e931332006d648b413e3d1c89f456a6b0dae1ab2e542ebad9416d483e4 Nov 04 05:27:45.053374 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:45.053351 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-788f8bd759-f6zjh"] Nov 04 05:27:45.458548 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:45.458511 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-788f8bd759-f6zjh" event={"ID":"11b27b07-7ada-441c-8799-e3d3d111bc18","Type":"ContainerStarted","Data":"7af0d5e931332006d648b413e3d1c89f456a6b0dae1ab2e542ebad9416d483e4"} Nov 04 05:27:46.024093 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:46.024005 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 05:27:46.026955 ip-10-0-130-185 kubenswrapper[2575]: I1104 05:27:46.026932 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:46.048839 ip-10-0-130-185 systemd[1]: Stopping Kubernetes Kubelet... Nov 04 05:27:46.060565 ip-10-0-130-185 systemd[1]: kubelet.service: Deactivated successfully. Nov 04 05:27:46.060790 ip-10-0-130-185 systemd[1]: Stopped Kubernetes Kubelet. Nov 04 05:27:46.061063 ip-10-0-130-185 systemd[1]: kubelet.service: Consumed 3.695s CPU time. Nov 04 05:27:46.070642 ip-10-0-130-185 systemd[1]: Starting Kubernetes Kubelet... Nov 04 05:27:46.166971 ip-10-0-130-185 kubenswrapper[5258]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:27:46.166971 ip-10-0-130-185 kubenswrapper[5258]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Nov 04 05:27:46.166971 ip-10-0-130-185 kubenswrapper[5258]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:27:46.166971 ip-10-0-130-185 kubenswrapper[5258]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 05:27:46.169426 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169335 5258 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:46.169426 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169423 5258 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:46.169426 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169426 5258 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:46.169426 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169430 5258 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:46.169426 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169432 5258 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:46.169426 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169435 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:46.169710 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169440 5258 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:46.169710 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169457 5258 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:46.169710 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169464 5258 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:46.169710 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169470 5258 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:46.169710 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169474 5258 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:46.169710 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169479 5258 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:46.169710 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169484 5258 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:46.169710 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169488 5258 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:46.169710 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169492 5258 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:46.169710 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169497 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:46.169710 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169502 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:46.169710 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169506 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:46.169710 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169511 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:46.169710 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169515 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:46.169710 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169519 5258 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:46.169710 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169522 5258 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:46.169710 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169525 5258 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:46.169710 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169529 5258 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:46.169710 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169531 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:46.170373 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169534 5258 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:46.170373 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169537 5258 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:46.170373 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169540 5258 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:46.170373 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169542 5258 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:46.170373 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169545 5258 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:46.170373 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169548 5258 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:46.170373 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169551 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:46.170373 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169554 5258 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:46.170373 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169557 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:46.170373 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169559 5258 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:46.170373 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169562 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:46.170373 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169565 5258 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:46.170373 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169568 5258 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:46.170373 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169570 5258 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:46.170373 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169574 5258 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:46.170373 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169577 5258 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:46.170373 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169579 5258 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:46.170373 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169582 5258 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:46.170373 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169584 5258 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:46.170373 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169587 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:46.171022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169589 5258 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:46.171022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169592 5258 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:46.171022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169595 5258 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:46.171022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169597 5258 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:46.171022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169600 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:46.171022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169603 5258 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:46.171022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169606 5258 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:46.171022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169608 5258 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:46.171022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169611 5258 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:46.171022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169614 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:46.171022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169617 5258 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:46.171022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169619 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:46.171022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169622 5258 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:46.171022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169626 5258 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:46.171022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169630 5258 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:46.171022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169635 5258 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:46.171022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169640 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:46.171022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169644 5258 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:46.171022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169651 5258 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:46.171676 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169656 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:46.171676 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169660 5258 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:46.171676 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169664 5258 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:46.171676 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169669 5258 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:46.171676 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169674 5258 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:46.171676 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169681 5258 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:46.171676 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169686 5258 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:46.171676 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169690 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:46.171676 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169698 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:46.171676 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169702 5258 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:46.171676 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169705 5258 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:46.171676 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169707 5258 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:46.171676 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169710 5258 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:46.171676 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169713 5258 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:46.171676 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169716 5258 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:46.171676 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169719 5258 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:46.171676 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169721 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:46.171676 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169725 5258 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:46.171676 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169728 5258 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:46.171676 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169730 5258 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:46.172364 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169733 5258 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:46.172364 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169736 5258 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:46.172364 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169738 5258 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:46.172364 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169741 5258 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:46.172364 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169744 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:46.172364 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169747 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:46.172364 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169749 5258 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:46.172364 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169752 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:46.172364 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169754 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:46.172364 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169757 5258 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:46.172364 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169760 5258 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:46.172364 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169764 5258 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:46.172364 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169766 5258 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:46.172364 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169769 5258 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:46.172364 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169772 5258 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:46.172364 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169775 5258 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:46.172364 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169778 5258 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:46.172364 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169780 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:46.172364 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169783 5258 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:46.173067 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169788 5258 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:46.173067 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169792 5258 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:46.173067 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.169808 5258 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:46.173067 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170746 5258 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:46.173067 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170756 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:46.173067 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170762 5258 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:46.173067 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170767 5258 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:46.173067 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170770 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:46.173067 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170773 5258 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:46.173067 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170776 5258 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:46.173067 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170780 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:46.173067 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170784 5258 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:46.173067 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170788 5258 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:46.173067 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170792 5258 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:46.173067 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170797 5258 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:46.173067 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170802 5258 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:46.173067 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170806 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:46.173067 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170811 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:46.173067 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170816 5258 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:46.173067 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170820 5258 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:46.173746 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170824 5258 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:46.173746 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170829 5258 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:46.173746 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170833 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:46.173746 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170837 5258 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:46.173746 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170845 5258 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:46.173746 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170851 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:46.173746 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170855 5258 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:46.173746 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170860 5258 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:46.173746 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170864 5258 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:46.173746 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170868 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:46.173746 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170872 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:46.173746 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170876 5258 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:46.173746 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170881 5258 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:46.173746 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170885 5258 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:46.173746 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170887 5258 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:46.173746 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170891 5258 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:46.173746 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170894 5258 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:46.173746 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170897 5258 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:46.173746 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170900 5258 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:46.174316 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170903 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:46.174316 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170906 5258 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:46.174316 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170908 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:46.174316 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170911 5258 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:46.174316 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170913 5258 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:46.174316 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170918 5258 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:46.174316 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170922 5258 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:46.174316 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170926 5258 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:46.174316 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170930 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:46.174316 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170934 5258 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:46.174316 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170938 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:46.174316 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170943 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:46.174316 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170948 5258 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:46.174316 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170955 5258 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:46.174316 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170961 5258 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:46.174316 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170966 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:46.174316 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170970 5258 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:46.174316 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170975 5258 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:46.174316 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170979 5258 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:46.174860 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170983 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:46.174860 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170986 5258 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:46.174860 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170989 5258 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:46.174860 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170991 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:46.174860 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.170996 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:46.174860 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171000 5258 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:46.174860 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171005 5258 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:46.174860 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171009 5258 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:46.174860 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171013 5258 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:46.174860 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171017 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:46.174860 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171022 5258 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:46.174860 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171027 5258 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:46.174860 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171045 5258 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:46.174860 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171050 5258 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:46.174860 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171054 5258 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:46.174860 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171058 5258 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:46.174860 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171062 5258 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:46.174860 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171066 5258 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:46.174860 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171071 5258 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:46.175492 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171075 5258 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:46.175492 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171077 5258 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:46.175492 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171080 5258 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:46.175492 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171083 5258 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:46.175492 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171085 5258 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:46.175492 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171089 5258 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:46.175492 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171091 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:46.175492 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171094 5258 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:46.175492 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171097 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:46.175492 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171099 5258 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:46.175492 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171104 5258 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:46.175492 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171108 5258 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:46.175492 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171112 5258 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:46.175492 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171117 5258 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:46.175492 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171121 5258 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:46.175492 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171126 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:46.175492 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171130 5258 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:46.175492 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171136 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:46.175492 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171140 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:46.175492 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171145 5258 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:46.176115 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171149 5258 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:46.176115 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171154 5258 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:46.176115 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171158 5258 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:46.176115 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171163 5258 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:46.176115 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171168 5258 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:46.176115 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171171 5258 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:46.176115 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171173 5258 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:46.176115 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171176 5258 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:46.176115 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171180 5258 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:46.176115 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171183 5258 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:46.176115 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171185 5258 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:46.176115 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171192 5258 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:46.176115 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171839 5258 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:46.176115 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171846 5258 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:46.176115 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171849 5258 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:46.176115 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171852 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:46.176115 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171855 5258 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:46.176115 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171858 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:46.176115 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171860 5258 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:46.176115 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171863 5258 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:46.176818 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171866 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:46.176818 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171869 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:46.176818 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171872 5258 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:46.176818 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171875 5258 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:46.176818 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171879 5258 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:46.176818 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171883 5258 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:46.176818 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171887 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:46.176818 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171891 5258 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:46.176818 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171894 5258 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:46.176818 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171897 5258 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:46.176818 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171902 5258 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:46.176818 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171906 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:46.176818 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171911 5258 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:46.176818 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171915 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:46.176818 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171919 5258 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:46.176818 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171922 5258 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:46.176818 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171926 5258 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:46.176818 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171928 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:46.176818 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171931 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:46.177442 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171934 5258 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:46.177442 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171936 5258 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:46.177442 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171939 5258 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:46.177442 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171941 5258 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:46.177442 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171946 5258 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:46.177442 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171949 5258 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:46.177442 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171952 5258 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:46.177442 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171956 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:46.177442 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171962 5258 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:46.177442 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171967 5258 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:46.177442 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171971 5258 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:46.177442 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171973 5258 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:46.177442 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171976 5258 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:46.177442 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171979 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:46.177442 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171982 5258 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:46.177442 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171986 5258 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:46.177442 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171992 5258 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:46.177442 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.171997 5258 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:46.177442 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172002 5258 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:46.178089 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172007 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:46.178089 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172011 5258 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:46.178089 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172020 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:46.178089 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172024 5258 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:46.178089 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172041 5258 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:46.178089 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172046 5258 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:46.178089 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172050 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:46.178089 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172054 5258 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:46.178089 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172059 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:46.178089 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172063 5258 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:46.178089 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172067 5258 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:46.178089 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172072 5258 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:46.178089 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172076 5258 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:46.178089 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172081 5258 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:46.178089 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172085 5258 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:46.178089 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172088 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:46.178089 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172091 5258 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:46.178089 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172095 5258 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:46.178089 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172099 5258 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:46.178089 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172103 5258 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:46.178914 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172108 5258 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:46.178914 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172114 5258 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:46.178914 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172118 5258 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:46.178914 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172124 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:46.178914 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172131 5258 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:46.178914 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172139 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:46.178914 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172145 5258 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:46.178914 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172150 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:46.178914 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172155 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:46.178914 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172159 5258 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:46.178914 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172164 5258 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:46.178914 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172169 5258 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:46.178914 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172174 5258 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:46.178914 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172179 5258 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:46.178914 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172183 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:46.178914 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172187 5258 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:46.178914 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172191 5258 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:46.178914 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172195 5258 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:46.178914 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172200 5258 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:46.179693 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172204 5258 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:46.179693 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172209 5258 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:46.179693 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172214 5258 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:46.179693 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172217 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:46.179693 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172221 5258 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:46.179693 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172227 5258 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:46.179693 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172233 5258 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:46.179693 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172238 5258 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:46.179693 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172242 5258 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:46.179693 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172246 5258 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:46.179693 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172252 5258 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:46.179693 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172255 5258 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:46.179693 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172258 5258 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:46.179693 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172261 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:46.179693 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172265 5258 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:46.179693 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172267 5258 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:46.179693 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172270 5258 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:46.179693 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172273 5258 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:46.179693 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172275 5258 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:46.179693 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172279 5258 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.172281 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172391 5258 flags.go:64] FLAG: --address="0.0.0.0" Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172401 5258 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172410 5258 flags.go:64] FLAG: --anonymous-auth="true" Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172417 5258 flags.go:64] FLAG: --application-metrics-count-limit="100" Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172421 5258 flags.go:64] FLAG: --authentication-token-webhook="false" Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172425 5258 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172429 5258 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172433 5258 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172436 5258 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172439 5258 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172443 5258 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172447 5258 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172450 5258 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172453 5258 flags.go:64] FLAG: --cgroup-root="" Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172456 5258 flags.go:64] FLAG: --cgroups-per-qos="true" Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172460 5258 flags.go:64] FLAG: --client-ca-file="" Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172465 5258 flags.go:64] FLAG: --cloud-provider="external" Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172470 5258 flags.go:64] FLAG: --cluster-dns="[]" Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172477 5258 flags.go:64] FLAG: --cluster-domain="" Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172482 5258 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172487 5258 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172496 5258 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Nov 04 05:27:46.180592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172502 5258 flags.go:64] FLAG: --container-log-max-files="5" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172509 5258 flags.go:64] FLAG: --container-log-max-size="10Mi" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172514 5258 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172522 5258 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172526 5258 flags.go:64] FLAG: --containerd-namespace="k8s.io" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172529 5258 flags.go:64] FLAG: --contention-profiling="false" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172532 5258 flags.go:64] FLAG: --cpu-cfs-quota="true" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172535 5258 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172538 5258 flags.go:64] FLAG: --cpu-manager-policy="none" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172541 5258 flags.go:64] FLAG: --cpu-manager-policy-options="" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172546 5258 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172549 5258 flags.go:64] FLAG: --enable-controller-attach-detach="true" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172552 5258 flags.go:64] FLAG: --enable-debugging-handlers="true" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172555 5258 flags.go:64] FLAG: --enable-load-reader="false" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172558 5258 flags.go:64] FLAG: --enable-server="true" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172561 5258 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172566 5258 flags.go:64] FLAG: --event-burst="100" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172569 5258 flags.go:64] FLAG: --event-qps="50" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172574 5258 flags.go:64] FLAG: --event-storage-age-limit="default=0" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172578 5258 flags.go:64] FLAG: --event-storage-event-limit="default=0" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172584 5258 flags.go:64] FLAG: --eviction-hard="" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172590 5258 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172594 5258 flags.go:64] FLAG: --eviction-minimum-reclaim="" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172599 5258 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172604 5258 flags.go:64] FLAG: --eviction-soft="" Nov 04 05:27:46.181686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172609 5258 flags.go:64] FLAG: --eviction-soft-grace-period="" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172614 5258 flags.go:64] FLAG: --exit-on-lock-contention="false" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172618 5258 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172622 5258 flags.go:64] FLAG: --experimental-mounter-path="" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172625 5258 flags.go:64] FLAG: --fail-cgroupv1="true" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172630 5258 flags.go:64] FLAG: --fail-swap-on="true" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172636 5258 flags.go:64] FLAG: --feature-gates="" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172642 5258 flags.go:64] FLAG: --file-check-frequency="20s" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172647 5258 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172652 5258 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172657 5258 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172660 5258 flags.go:64] FLAG: --healthz-port="10248" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172663 5258 flags.go:64] FLAG: --help="false" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172666 5258 flags.go:64] FLAG: --hostname-override="ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172669 5258 flags.go:64] FLAG: --housekeeping-interval="10s" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172673 5258 flags.go:64] FLAG: --http-check-frequency="20s" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172712 5258 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172718 5258 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172723 5258 flags.go:64] FLAG: --image-gc-high-threshold="85" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172728 5258 flags.go:64] FLAG: --image-gc-low-threshold="80" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172732 5258 flags.go:64] FLAG: --image-service-endpoint="" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172737 5258 flags.go:64] FLAG: --kernel-memcg-notification="false" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172747 5258 flags.go:64] FLAG: --kube-api-burst="100" Nov 04 05:27:46.182801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172751 5258 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172755 5258 flags.go:64] FLAG: --kube-api-qps="50" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172760 5258 flags.go:64] FLAG: --kube-reserved="" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172765 5258 flags.go:64] FLAG: --kube-reserved-cgroup="" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172769 5258 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172774 5258 flags.go:64] FLAG: --kubelet-cgroups="" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172779 5258 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172784 5258 flags.go:64] FLAG: --lock-file="" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172788 5258 flags.go:64] FLAG: --log-cadvisor-usage="false" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172794 5258 flags.go:64] FLAG: --log-flush-frequency="5s" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172799 5258 flags.go:64] FLAG: --log-json-info-buffer-size="0" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172808 5258 flags.go:64] FLAG: --log-json-split-stream="false" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172818 5258 flags.go:64] FLAG: --log-text-info-buffer-size="0" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172824 5258 flags.go:64] FLAG: --log-text-split-stream="false" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172827 5258 flags.go:64] FLAG: --logging-format="text" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172832 5258 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172836 5258 flags.go:64] FLAG: --make-iptables-util-chains="true" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172839 5258 flags.go:64] FLAG: --manifest-url="" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172843 5258 flags.go:64] FLAG: --manifest-url-header="" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172859 5258 flags.go:64] FLAG: --max-housekeeping-interval="15s" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172862 5258 flags.go:64] FLAG: --max-open-files="1000000" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172866 5258 flags.go:64] FLAG: --max-pods="110" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172869 5258 flags.go:64] FLAG: --maximum-dead-containers="-1" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172873 5258 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172876 5258 flags.go:64] FLAG: --memory-manager-policy="None" Nov 04 05:27:46.183882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172879 5258 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172882 5258 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172885 5258 flags.go:64] FLAG: --node-ip="0.0.0.0" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172888 5258 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172896 5258 flags.go:64] FLAG: --node-status-max-images="50" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172899 5258 flags.go:64] FLAG: --node-status-update-frequency="10s" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172903 5258 flags.go:64] FLAG: --oom-score-adj="-999" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172906 5258 flags.go:64] FLAG: --pod-cidr="" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172908 5258 flags.go:64] FLAG: --pod-manifest-path="" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172911 5258 flags.go:64] FLAG: --pod-max-pids="-1" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172914 5258 flags.go:64] FLAG: --pods-per-core="0" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172917 5258 flags.go:64] FLAG: --port="10250" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172920 5258 flags.go:64] FLAG: --protect-kernel-defaults="false" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172923 5258 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-0da4f6eccf68ffd9d" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172926 5258 flags.go:64] FLAG: --qos-reserved="" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172930 5258 flags.go:64] FLAG: --read-only-port="10255" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172933 5258 flags.go:64] FLAG: --register-node="true" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172935 5258 flags.go:64] FLAG: --register-with-taints="" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172939 5258 flags.go:64] FLAG: --registry-burst="10" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172942 5258 flags.go:64] FLAG: --registry-qps="5" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172945 5258 flags.go:64] FLAG: --reserved-cpus="" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172949 5258 flags.go:64] FLAG: --reserved-memory="" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172958 5258 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172961 5258 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172964 5258 flags.go:64] FLAG: --rotate-certificates="false" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172968 5258 flags.go:64] FLAG: --rotate-server-certificates="false" Nov 04 05:27:46.185023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172972 5258 flags.go:64] FLAG: --runonce="false" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172975 5258 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172979 5258 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172982 5258 flags.go:64] FLAG: --seccomp-default="false" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172985 5258 flags.go:64] FLAG: --serialize-image-pulls="true" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172988 5258 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172991 5258 flags.go:64] FLAG: --storage-driver-db="cadvisor" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172994 5258 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.172998 5258 flags.go:64] FLAG: --storage-driver-password="root" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.173002 5258 flags.go:64] FLAG: --storage-driver-secure="false" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.173005 5258 flags.go:64] FLAG: --storage-driver-table="stats" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.173008 5258 flags.go:64] FLAG: --storage-driver-user="root" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.173011 5258 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.173014 5258 flags.go:64] FLAG: --sync-frequency="1m0s" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.173017 5258 flags.go:64] FLAG: --system-cgroups="" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.173020 5258 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.173025 5258 flags.go:64] FLAG: --system-reserved-cgroup="" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.173028 5258 flags.go:64] FLAG: --tls-cert-file="" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.173050 5258 flags.go:64] FLAG: --tls-cipher-suites="[]" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.173056 5258 flags.go:64] FLAG: --tls-min-version="" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.173061 5258 flags.go:64] FLAG: --tls-private-key-file="" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.173066 5258 flags.go:64] FLAG: --topology-manager-policy="none" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.173069 5258 flags.go:64] FLAG: --topology-manager-policy-options="" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.173072 5258 flags.go:64] FLAG: --topology-manager-scope="container" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.173075 5258 flags.go:64] FLAG: --v="2" Nov 04 05:27:46.186181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.173080 5258 flags.go:64] FLAG: --version="false" Nov 04 05:27:46.187262 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.173085 5258 flags.go:64] FLAG: --vmodule="" Nov 04 05:27:46.187262 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.173089 5258 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Nov 04 05:27:46.187262 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.173094 5258 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Nov 04 05:27:46.187262 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173272 5258 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:46.187262 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173277 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:46.187262 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173280 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:46.187262 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173287 5258 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:46.187262 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173292 5258 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:46.187262 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173296 5258 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:46.187262 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173300 5258 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:46.187262 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173305 5258 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:46.187262 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173309 5258 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:46.187262 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173312 5258 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:46.187262 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173317 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:46.187262 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173322 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:46.187262 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173325 5258 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:46.187262 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173328 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:46.187262 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173331 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:46.187262 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173334 5258 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:46.187262 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173337 5258 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:46.188022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173339 5258 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:46.188022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173342 5258 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:46.188022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173345 5258 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:46.188022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173347 5258 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:46.188022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173350 5258 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:46.188022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173352 5258 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:46.188022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173355 5258 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:46.188022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173357 5258 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:46.188022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173360 5258 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:46.188022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173362 5258 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:46.188022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173366 5258 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:46.188022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173370 5258 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:46.188022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173373 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:46.188022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173375 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:46.188022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173378 5258 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:46.188022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173382 5258 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:46.188022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173385 5258 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:46.188022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173387 5258 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:46.188022 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173392 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:46.188724 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173394 5258 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:46.188724 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173397 5258 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:46.188724 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173399 5258 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:46.188724 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173402 5258 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:46.188724 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173404 5258 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:46.188724 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173407 5258 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:46.188724 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173409 5258 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:46.188724 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173412 5258 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:46.188724 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173415 5258 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:46.188724 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173417 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:46.188724 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173420 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:46.188724 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173423 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:46.188724 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173425 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:46.188724 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173428 5258 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:46.188724 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173431 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:46.188724 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173433 5258 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:46.188724 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173436 5258 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:46.188724 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173438 5258 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:46.188724 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173441 5258 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:46.188724 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173443 5258 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:46.189462 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173446 5258 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:46.189462 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173450 5258 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:46.189462 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173454 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:46.189462 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173458 5258 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:46.189462 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173461 5258 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:46.189462 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173465 5258 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:46.189462 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173469 5258 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:46.189462 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173473 5258 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:46.189462 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173477 5258 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:46.189462 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173482 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:46.189462 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173486 5258 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:46.189462 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173490 5258 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:46.189462 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173493 5258 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:46.189462 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173495 5258 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:46.189462 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173498 5258 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:46.189462 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173501 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:46.189462 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173503 5258 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:46.189462 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173507 5258 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:46.189462 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173511 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:46.190344 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173514 5258 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:46.190344 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173517 5258 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:46.190344 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173519 5258 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:46.190344 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173522 5258 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:46.190344 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173525 5258 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:46.190344 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173527 5258 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:46.190344 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173530 5258 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:46.190344 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173533 5258 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:46.190344 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173535 5258 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:46.190344 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173538 5258 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:46.190344 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173540 5258 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:46.190344 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173543 5258 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:46.190344 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173545 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:46.190344 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173548 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:46.190344 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173550 5258 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:46.190344 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173553 5258 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:46.190344 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173555 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:46.190344 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173558 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:46.190344 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173561 5258 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:46.190344 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173563 5258 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:46.191215 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173566 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:46.191215 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173568 5258 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:46.191215 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173570 5258 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:46.191215 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173573 5258 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:46.191215 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173577 5258 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:46.191215 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173580 5258 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:46.191215 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173583 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:46.191215 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173585 5258 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:46.191215 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173588 5258 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:46.191215 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173591 5258 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:46.191215 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173595 5258 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:46.191215 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.173633 5258 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:27:46.191215 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173684 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:46.191215 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173690 5258 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:46.191215 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173693 5258 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:46.191215 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173696 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:46.191840 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173699 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:46.191840 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173702 5258 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:46.191840 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173706 5258 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:46.191840 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173709 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:46.191840 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173711 5258 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:46.191840 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173715 5258 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:46.191840 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173718 5258 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:46.191840 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173720 5258 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:46.191840 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173723 5258 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:46.191840 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173726 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:46.191840 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173729 5258 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:46.191840 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173732 5258 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:46.191840 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173734 5258 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:46.191840 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173737 5258 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:46.191840 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173740 5258 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:46.191840 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173742 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:46.191840 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173745 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:46.191840 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173748 5258 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:46.191840 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173750 5258 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:46.192595 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173753 5258 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:46.192595 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173758 5258 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:46.192595 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173761 5258 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:46.192595 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173764 5258 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:46.192595 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173766 5258 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:46.192595 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173769 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:46.192595 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173772 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:46.192595 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173776 5258 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:46.192595 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173782 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:46.192595 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173786 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:46.192595 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173790 5258 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:46.192595 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173794 5258 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:46.192595 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173799 5258 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:46.192595 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173803 5258 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:46.192595 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173806 5258 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:46.192595 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173810 5258 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:46.192595 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173814 5258 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:46.192595 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173816 5258 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:46.192595 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173819 5258 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:46.192595 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173822 5258 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:46.193261 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173824 5258 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:46.193261 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173827 5258 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:46.193261 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173830 5258 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:46.193261 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173835 5258 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:46.193261 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173837 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:46.193261 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173840 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:46.193261 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173844 5258 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:46.193261 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173849 5258 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:46.193261 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173854 5258 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:46.193261 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173858 5258 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:46.193261 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173862 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:46.193261 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173866 5258 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:46.193261 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173872 5258 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:46.193261 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173881 5258 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:46.193261 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173886 5258 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:46.193261 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173889 5258 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:46.193261 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173892 5258 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:46.193261 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173894 5258 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:46.193261 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173898 5258 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:46.193920 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173901 5258 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:46.193920 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173906 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:46.193920 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173910 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:46.193920 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173914 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:46.193920 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173918 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:46.193920 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173923 5258 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:46.193920 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173927 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:46.193920 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173931 5258 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:46.193920 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173935 5258 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:46.193920 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173940 5258 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:46.193920 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173944 5258 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:46.193920 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173947 5258 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:46.193920 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173949 5258 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:46.193920 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173952 5258 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:46.193920 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173955 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:46.193920 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173957 5258 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:46.193920 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173960 5258 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:46.193920 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173964 5258 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:46.193920 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173968 5258 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:46.194512 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173973 5258 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:46.194512 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173977 5258 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:46.194512 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173982 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:46.194512 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173987 5258 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:46.194512 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173993 5258 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:46.194512 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.173997 5258 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:46.194512 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.174000 5258 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:46.194512 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.174004 5258 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:46.194512 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.174007 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:46.194512 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.174013 5258 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:46.194512 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.174018 5258 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:46.194512 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.174022 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:46.194512 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.174025 5258 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:46.194512 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.174043 5258 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:46.194512 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.174051 5258 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:46.194512 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.174055 5258 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:46.194512 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.174059 5258 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:46.194512 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.174063 5258 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:46.194512 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.174067 5258 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:46.195016 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.174071 5258 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:46.195016 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.174075 5258 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:46.195016 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.174079 5258 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:46.195016 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.174084 5258 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:46.195016 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.174088 5258 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:46.195016 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.174091 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:46.195016 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.178839 5258 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Nov 04 05:27:46.195016 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.178855 5258 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Nov 04 05:27:46.195016 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.178875 5258 watchdog_linux.go:95] "Systemd watchdog is not enabled" Nov 04 05:27:46.195016 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.178880 5258 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Nov 04 05:27:46.195016 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.178995 5258 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:46.195016 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179001 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:46.195016 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179005 5258 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:46.195016 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179009 5258 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:46.195016 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179012 5258 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:46.195016 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179015 5258 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:46.195016 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179018 5258 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:46.195016 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179021 5258 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:46.195016 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179024 5258 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:46.195805 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179029 5258 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:46.195805 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179055 5258 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:46.195805 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179062 5258 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:46.195805 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179066 5258 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:46.195805 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179069 5258 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:46.195805 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179073 5258 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:46.195805 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179077 5258 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:46.195805 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179081 5258 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:46.195805 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179084 5258 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:46.195805 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179088 5258 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:46.195805 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179091 5258 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:46.195805 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179094 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:46.195805 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179096 5258 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:46.195805 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179099 5258 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:46.195805 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179102 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:46.195805 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179108 5258 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:46.195805 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179111 5258 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:46.195805 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179114 5258 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:46.195805 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179117 5258 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:46.196566 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179120 5258 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:46.196566 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179141 5258 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:46.196566 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179145 5258 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:46.196566 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179148 5258 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:46.196566 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179151 5258 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:46.196566 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179154 5258 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:46.196566 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179157 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:46.196566 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179160 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:46.196566 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179166 5258 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:46.196566 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179169 5258 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:46.196566 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179172 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:46.196566 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179175 5258 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:46.196566 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179178 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:46.196566 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179181 5258 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:46.196566 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179184 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:46.196566 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179187 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:46.196566 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179190 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:46.196566 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179193 5258 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:46.196566 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179196 5258 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:46.196566 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179199 5258 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:46.197285 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179202 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:46.197285 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179208 5258 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:46.197285 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179211 5258 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:46.197285 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179214 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:46.197285 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179217 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:46.197285 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179220 5258 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:46.197285 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179223 5258 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:46.197285 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179225 5258 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:46.197285 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179228 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:46.197285 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179231 5258 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:46.197285 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179234 5258 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:46.197285 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179237 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:46.197285 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179240 5258 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:46.197285 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179243 5258 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:46.197285 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179266 5258 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:46.197285 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179269 5258 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:46.197285 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179272 5258 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:46.197285 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179275 5258 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:46.197285 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179277 5258 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:46.197934 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179280 5258 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:46.197934 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179283 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:46.197934 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179286 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:46.197934 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179289 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:46.197934 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179291 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:46.197934 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179294 5258 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:46.197934 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179297 5258 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:46.197934 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179304 5258 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:46.197934 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179307 5258 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:46.197934 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179310 5258 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:46.197934 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179312 5258 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:46.197934 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179315 5258 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:46.197934 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179318 5258 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:46.197934 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179321 5258 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:46.197934 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179324 5258 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:46.197934 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179326 5258 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:46.197934 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179329 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:46.197934 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179332 5258 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:46.197934 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179335 5258 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:46.197934 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179338 5258 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:46.198713 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179343 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:46.198713 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179346 5258 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:46.198713 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179349 5258 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:46.198713 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179361 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:46.198713 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179364 5258 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:46.198713 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179367 5258 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:46.198713 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179369 5258 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:46.198713 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179372 5258 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:46.198713 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179391 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:46.198713 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179395 5258 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:46.198713 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179398 5258 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:46.198713 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179401 5258 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:46.198713 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179403 5258 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:46.198713 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179409 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:46.198713 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179412 5258 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:46.198713 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179415 5258 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:46.198713 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179418 5258 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:46.198713 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179420 5258 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:46.198713 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.179423 5258 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:46.199348 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.179464 5258 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:27:46.199348 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180003 5258 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:46.199348 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180020 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:46.199348 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180028 5258 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:46.199348 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180050 5258 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:46.199348 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180055 5258 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:46.199348 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180072 5258 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:46.199348 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180077 5258 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:46.199348 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180082 5258 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:46.199348 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180086 5258 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:46.199348 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180091 5258 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:46.199348 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180096 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:46.199348 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180100 5258 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:46.199348 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180105 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:46.199348 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180109 5258 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:46.199348 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180114 5258 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:46.199931 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180119 5258 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:46.199931 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180123 5258 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:46.199931 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180128 5258 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:46.199931 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180137 5258 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:46.199931 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180142 5258 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:46.199931 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180147 5258 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:46.199931 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180152 5258 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:46.199931 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180156 5258 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:46.199931 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180161 5258 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:46.199931 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180165 5258 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:46.199931 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180169 5258 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:46.199931 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180174 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:46.199931 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180179 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:46.199931 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180184 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:46.199931 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180189 5258 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:46.199931 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180194 5258 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:46.199931 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180203 5258 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:46.199931 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180208 5258 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:46.199931 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180214 5258 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:46.200654 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180218 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:46.200654 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180223 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:46.200654 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180228 5258 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:46.200654 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180232 5258 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:46.200654 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180414 5258 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:46.200654 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180420 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:46.200654 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180426 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:46.200654 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180431 5258 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:46.200654 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180436 5258 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:46.200654 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180440 5258 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:46.200654 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180450 5258 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:46.200654 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180456 5258 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:46.200654 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180461 5258 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:46.200654 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180465 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:46.200654 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180471 5258 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:46.200654 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180476 5258 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:46.200654 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180481 5258 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:46.200654 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180485 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:46.200654 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180490 5258 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:46.200654 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180494 5258 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:46.201438 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180499 5258 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:46.201438 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180504 5258 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:46.201438 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180515 5258 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:46.201438 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180519 5258 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:46.201438 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180524 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:46.201438 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180528 5258 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:46.201438 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180535 5258 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:46.201438 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180542 5258 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:46.201438 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180547 5258 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:46.201438 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180555 5258 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:46.201438 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180563 5258 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:46.201438 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180568 5258 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:46.201438 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180575 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:46.201438 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180580 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:46.201438 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180589 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:46.201438 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180593 5258 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:46.201438 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180597 5258 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:46.201438 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180602 5258 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:46.201438 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180607 5258 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:46.202161 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180611 5258 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:46.202161 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180615 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:46.202161 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180619 5258 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:46.202161 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180624 5258 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:46.202161 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180628 5258 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:46.202161 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180633 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:46.202161 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180638 5258 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:46.202161 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180643 5258 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:46.202161 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180652 5258 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:46.202161 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180657 5258 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:46.202161 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180662 5258 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:46.202161 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180667 5258 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:46.202161 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180671 5258 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:46.202161 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180676 5258 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:46.202161 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180681 5258 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:46.202161 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180686 5258 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:46.202161 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180690 5258 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:46.202161 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180756 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:46.202161 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180763 5258 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:46.202161 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180769 5258 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:46.202875 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180778 5258 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:46.202875 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180784 5258 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:46.202875 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180789 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:46.202875 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180793 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:46.202875 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180798 5258 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:46.202875 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180803 5258 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:46.202875 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180808 5258 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:46.202875 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180813 5258 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:46.202875 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180819 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:46.202875 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180823 5258 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:46.202875 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180828 5258 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:46.202875 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180832 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:46.202875 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.180837 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:46.202875 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.180916 5258 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 05:27:46.202875 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181060 5258 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 05:27:46.203464 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181069 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 05:27:46.203464 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181074 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 05:27:46.203464 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181085 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 05:27:46.203464 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181090 5258 feature_gate.go:386] unrecognized feature gate: Example Nov 04 05:27:46.203464 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181095 5258 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 05:27:46.203464 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181100 5258 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 05:27:46.203464 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181104 5258 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 05:27:46.203464 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181109 5258 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 05:27:46.203464 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181113 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 05:27:46.203464 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181118 5258 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 05:27:46.203464 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181123 5258 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 05:27:46.203464 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181127 5258 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 05:27:46.203464 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181132 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 05:27:46.203464 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181137 5258 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 05:27:46.203464 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181146 5258 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 05:27:46.203464 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181151 5258 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 05:27:46.203464 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181155 5258 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 05:27:46.203464 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181160 5258 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 05:27:46.203464 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181164 5258 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 05:27:46.203464 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181168 5258 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 05:27:46.204385 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181173 5258 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 05:27:46.204385 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181178 5258 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 05:27:46.204385 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181183 5258 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 05:27:46.204385 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181187 5258 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 05:27:46.204385 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181193 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 05:27:46.204385 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181198 5258 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 05:27:46.204385 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181203 5258 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 05:27:46.204385 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181212 5258 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 05:27:46.204385 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181217 5258 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 05:27:46.204385 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181221 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 05:27:46.204385 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181227 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 05:27:46.204385 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181237 5258 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 05:27:46.204385 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181242 5258 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 05:27:46.204385 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181247 5258 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 05:27:46.204385 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181251 5258 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 05:27:46.204385 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181256 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 05:27:46.204385 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181261 5258 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 05:27:46.204385 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181265 5258 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 05:27:46.204385 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181270 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 05:27:46.205307 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181278 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 05:27:46.205307 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181283 5258 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 05:27:46.205307 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181287 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 05:27:46.205307 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181293 5258 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 05:27:46.205307 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181297 5258 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 05:27:46.205307 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181302 5258 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 05:27:46.205307 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181306 5258 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 05:27:46.205307 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181311 5258 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 05:27:46.205307 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181316 5258 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 05:27:46.205307 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181320 5258 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 05:27:46.205307 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181325 5258 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 05:27:46.205307 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181329 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 05:27:46.205307 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181334 5258 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 05:27:46.205307 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181343 5258 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 05:27:46.205307 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181348 5258 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 05:27:46.205307 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181353 5258 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 05:27:46.205307 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181357 5258 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 05:27:46.205307 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181362 5258 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 05:27:46.205307 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181367 5258 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 05:27:46.205307 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181375 5258 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 05:27:46.205942 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181380 5258 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 05:27:46.205942 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181384 5258 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 05:27:46.205942 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181389 5258 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 05:27:46.205942 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181394 5258 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 05:27:46.205942 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181398 5258 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 05:27:46.205942 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181403 5258 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 05:27:46.205942 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181412 5258 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 05:27:46.205942 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181416 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 05:27:46.205942 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181421 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 05:27:46.205942 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181425 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 05:27:46.205942 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181429 5258 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 05:27:46.205942 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181434 5258 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 05:27:46.205942 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181439 5258 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 05:27:46.205942 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181444 5258 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 05:27:46.205942 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181448 5258 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 05:27:46.205942 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181453 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 05:27:46.205942 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181457 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 05:27:46.205942 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181461 5258 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 05:27:46.205942 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181466 5258 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 05:27:46.206620 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181475 5258 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 05:27:46.206620 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181479 5258 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 05:27:46.206620 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181484 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 05:27:46.206620 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181488 5258 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 05:27:46.206620 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181493 5258 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 05:27:46.206620 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181497 5258 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 05:27:46.206620 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181501 5258 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 05:27:46.206620 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181509 5258 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 05:27:46.206620 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181513 5258 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 05:27:46.206620 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181518 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 05:27:46.206620 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181523 5258 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 05:27:46.206620 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181527 5258 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 05:27:46.206620 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181539 5258 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 05:27:46.206620 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181544 5258 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 05:27:46.206620 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181548 5258 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 05:27:46.206620 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181554 5258 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 05:27:46.206620 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181563 5258 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 05:27:46.206620 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181568 5258 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 05:27:46.206620 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181573 5258 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 05:27:46.206620 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181577 5258 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 05:27:46.207409 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181582 5258 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 05:27:46.207409 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181586 5258 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 05:27:46.207409 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181593 5258 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 05:27:46.207409 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181599 5258 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 05:27:46.207409 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181610 5258 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 05:27:46.207409 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181620 5258 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 05:27:46.207409 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:46.181626 5258 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 05:27:46.207409 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.181744 5258 server.go:957] "Client rotation is on, will bootstrap in background" Nov 04 05:27:46.207409 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.183867 5258 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Nov 04 05:27:46.207409 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.183965 5258 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Nov 04 05:27:46.207409 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.184336 5258 server.go:1014] "Starting client certificate rotation" Nov 04 05:27:46.207409 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.184425 5258 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 05:27:46.207409 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.184464 5258 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 05:21:32 +0000 UTC" deadline="2028-04-07 21:47:11.36084671 +0000 UTC" Nov 04 05:27:46.207409 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.184520 5258 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12496h19m25.176329663s" Nov 04 05:27:46.207409 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.185631 5258 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 05:27:46.207982 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.187325 5258 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 05:27:46.207982 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.188834 5258 log.go:25] "Validated CRI v1 runtime API" Nov 04 05:27:46.207982 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.191087 5258 log.go:25] "Validated CRI v1 image API" Nov 04 05:27:46.207982 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.191976 5258 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Nov 04 05:27:46.207982 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.194695 5258 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a3e82bfd-4010-41f9-be4c-a0e2c0261e88:/dev/nvme0n1p4 ff3690df-fbfb-4f34-b429-96733430703f:/dev/nvme0n1p3] Nov 04 05:27:46.208267 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.194718 5258 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/09f4f8137bae50dd458fac2cf6749649224d80f65839a31044896d947a39ed15/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/09f4f8137bae50dd458fac2cf6749649224d80f65839a31044896d947a39ed15/userdata/shm major:0 minor:306 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0a960f30f55e9ac08398126969050a8c92023b2d69187c0d0eea1a543fea9c06/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0a960f30f55e9ac08398126969050a8c92023b2d69187c0d0eea1a543fea9c06/userdata/shm major:0 minor:273 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1e1ee7a2fb796112615a893a9293f3886537d412fe3873c3445751d9253ff591/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1e1ee7a2fb796112615a893a9293f3886537d412fe3873c3445751d9253ff591/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1fa2fef846d263f0e570b3c8a9f936acd1a5f014e1a7f3ada3b50c735aa4d8f5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1fa2fef846d263f0e570b3c8a9f936acd1a5f014e1a7f3ada3b50c735aa4d8f5/userdata/shm major:0 minor:310 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/24cdb6fe97a2013caf78391931390bf69ab3ce4a9eaba0a900ce8608f2503d69/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/24cdb6fe97a2013caf78391931390bf69ab3ce4a9eaba0a900ce8608f2503d69/userdata/shm major:0 minor:93 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2a9d4ef75ba4781c64c7c7c60ddf779f1b146f6a366f618c4b12bdd284719c10/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2a9d4ef75ba4781c64c7c7c60ddf779f1b146f6a366f618c4b12bdd284719c10/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/32c7911aad73a69827aa71e308653a4a1881fca1a866a239e682ee0fb082b3b8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/32c7911aad73a69827aa71e308653a4a1881fca1a866a239e682ee0fb082b3b8/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/44e9081016a2d01003ff747c050f2c3c9f81da9cc01e11bbf8f204409360c56d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/44e9081016a2d01003ff747c050f2c3c9f81da9cc01e11bbf8f204409360c56d/userdata/shm major:0 minor:346 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/46d6e055c92ced70e25d887a457d36afe3656f32caae299a832604f3d72f46cb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/46d6e055c92ced70e25d887a457d36afe3656f32caae299a832604f3d72f46cb/userdata/shm major:0 minor:338 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4ef62602015af169a11d55bbd552150ac8a077a1aa4ca2fa69349c9de0faa96d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4ef62602015af169a11d55bbd552150ac8a077a1aa4ca2fa69349c9de0faa96d/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/77b95a272239a218daaf92e07e09e4b57be0328dc6885234f7911f53112e8631/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/77b95a272239a218daaf92e07e09e4b57be0328dc6885234f7911f53112e8631/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7af0d5e931332006d648b413e3d1c89f456a6b0dae1ab2e542ebad9416d483e4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7af0d5e931332006d648b413e3d1c89f456a6b0dae1ab2e542ebad9416d483e4/userdata/shm major:0 minor:370 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7dac931844b45988d66af94c90834d70aa8f08c60d881ddb099b21a1b925e5a0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7dac931844b45988d66af94c90834d70aa8f08c60d881ddb099b21a1b925e5a0/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7dd94360c70bdb8282c2497a257293d5cc50c73f8a0c7335f4f731319dac7186/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7dd94360c70bdb8282c2497a257293d5cc50c73f8a0c7335f4f731319dac7186/userdata/shm major:0 minor:81 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9d880bbb709757092439f940aec40765b05af6049d4904cba4b6734b167190a0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9d880bbb709757092439f940aec40765b05af6049d4904cba4b6734b167190a0/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a0e86e8bd29d3b22725f3b36e1864b50e3cd003841456838da9511ec22ba3da5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a0e86e8bd29d3b22725f3b36e1864b50e3cd003841456838da9511ec22ba3da5/userdata/shm major:0 minor:308 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a2cc34cd793b26d5679286cb22c1afd256a95096c2c83af5b04a0dc51c24bdcf/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a2cc34cd793b26d5679286cb22c1afd256a95096c2c83af5b04a0dc51c24bdcf/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b09efdaca27be91789cf702b91cff41442e4c9e4b102ee2c6463478c3f82e0b7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b09efdaca27be91789cf702b91cff41442e4c9e4b102ee2c6463478c3f82e0b7/userdata/shm major:0 minor:241 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b6c2dbf7b1476ea572b7848a417ec1412e9c6f2704f5e77696912d547c9e8c3e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b6c2dbf7b1476ea572b7848a417ec1412e9c6f2704f5e77696912d547c9e8c3e/userdata/shm major:0 minor:281 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c1b7a155d8e593e9cc5948587dcc48347fc8335dbd52a99eb517fbccf3cef547/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c1b7a155d8e593e9cc5948587dcc48347fc8335dbd52a99eb517fbccf3cef547/userdata/shm major:0 minor:213 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d353d62f4bcc28a17a12b87e478a93b66ce4b99738e4bd8066eef5b9ff8ddfc2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d353d62f4bcc28a17a12b87e478a93b66ce4b99738e4bd8066eef5b9ff8ddfc2/userdata/shm major:0 minor:348 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e06cf59f7386c12d3abbe5721afac2d9bb4ec36682b2747705eb514275347b6d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e06cf59f7386c12d3abbe5721afac2d9bb4ec36682b2747705eb514275347b6d/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e456f1812cccb3da24cd453c17ed3fc2f10fe153298688919214022cc2035c00/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e456f1812cccb3da24cd453c17ed3fc2f10fe153298688919214022cc2035c00/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f24431ce12e2fb36139b8e783f12f8402f9c33ee32afe71c6d090c111a9bb5b2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f24431ce12e2fb36139b8e783f12f8402f9c33ee32afe71c6d090c111a9bb5b2/userdata/shm major:0 minor:283 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fbaa1a138a1355400c58c2fbd44b7cd69611c765efe58c7e2ae239c8086b05c6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fbaa1a138a1355400c58c2fbd44b7cd69611c765efe58c7e2ae239c8086b05c6/userdata/shm major:0 minor:208 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/00c77006-a012-4f0e-a886-69ff0150de6d/volumes/kubernetes.io~projected/kube-api-access-6b7cc:{mountpoint:/var/lib/kubelet/pods/00c77006-a012-4f0e-a886-69ff0150de6d/volumes/kubernetes.io~projected/kube-api-access-6b7cc major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/11b27b07-7ada-441c-8799-e3d3d111bc18/volumes/kubernetes.io~projected/kube-api-access-s58tj:{mountpoint:/var/lib/kubelet/pods/11b27b07-7ada-441c-8799-e3d3d111bc18/volumes/kubernetes.io~projected/kube-api-access-s58tj major:0 minor:369 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/11b27b07-7ada-441c-8799-e3d3d111bc18/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/11b27b07-7ada-441c-8799-e3d3d111bc18/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:364 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/11b27b07-7ada-441c-8799-e3d3d111bc18/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/11b27b07-7ada-441c-8799-e3d3d111bc18/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:368 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/23cf3ade-4fac-44f1-813c-7f61427e2b83/volumes/kubernetes.io~projected/kube-api-access-r2bc5:{mountpoint:/var/lib/kubelet/pods/23cf3ade-4fac-44f1-813c-7f61427e2b83/volumes/kubernetes.io~projected/kube-api-access-r2bc5 major:0 minor:305 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4806f73f-d8d3-41a3-865f-9df7fe88e1b8/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/4806f73f-d8d3-41a3-865f-9df7fe88e1b8/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:278 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4806f73f-d8d3-41a3-865f-9df7fe88e1b8/volumes/kubernetes.io~projected/kube-api-access-2tcm2:{mountpoint:/var/lib/kubelet/pods/4806f73f-d8d3-41a3-865f-9df7fe88e1b8/volumes/kubernetes.io~projected/kube-api-access-2tcm2 major:0 minor:280 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4806f73f-d8d3-41a3-865f-9df7fe88e1b8/volumes/kubernetes.io~projected/registry-tls:{mountpoint:/var/lib/kubelet/pods/4806f73f-d8d3-41a3-865f-9df7fe88e1b8/volumes/kubernetes.io~projected/registry-tls major:0 minor:276 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4806f73f-d8d3-41a3-865f-9df7fe88e1b8/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/4806f73f-d8d3-41a3-865f-9df7fe88e1b8/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:277 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4806f73f-d8d3-41a3-865f-9df7fe88e1b8/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/4806f73f-d8d3-41a3-865f-9df7fe88e1b8/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:275 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a120e88-95d0-4176-ab05-586d4c2e7dab/volumes/kubernetes.io~projected/kube-api-access-bll88:{mountpoint:/var/lib/kubelet/pods/4a120e88-95d0-4176-ab05-586d4c2e7dab/volumes/kubernetes.io~projected/kube-api-access-bll88 major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4dcc3e21-060d-4d88-a034-025ee61af3ff/volumes/kubernetes.io~projected/kube-api-access-tvqg7:{mountpoint:/var/lib/kubelet/pods/4dcc3e21-060d-4d88-a034-025ee61af3ff/volumes/kubernetes.io~projected/kube-api-access-tvqg7 major:0 minor:337 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4dcc3e21-060d-4d88-a034-025ee61af3ff/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/4dcc3e21-060d-4d88-a034-025ee61af3ff/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:336 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4dcc3e21-060d-4d88-a034-025ee61af3ff/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/4dcc3e21-060d-4d88-a034-025ee61af3ff/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:331 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4fc979bb-e663-4465-a2cc-16cb06be4963/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/4fc979bb-e663-4465-a2cc-16cb06be4963/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4fc979bb-e663-4465-a2cc-16cb06be4963/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/4fc979bb-e663-4465-a2cc-16cb06be4963/volumes/kubernetes.io~empty-dir/tmp major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4fc979bb-e663-4465-a2cc-16cb06be4963/volumes/kubernetes.io~projected/kube-api-access-nzngz:{mountpoint:/var/lib/kubelet/pods/4fc979bb-e663-4465-a2cc-16cb06be4963/volumes/kubernetes.io~projected/kube-api-access-nzngz major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5a710da8-8925-4e16-b43d-fc06a999569b/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/5a710da8-8925-4e16-b43d-fc06a999569b/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:299 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5bb1115f-702d-4b53-8852-4b38e452edb6/volumes/kubernetes.io~projected/kube-api-access-rmnb5:{mountpoint:/var/lib/kubelet/pods/5bb1115f-702d-4b53-8852-4b38e452edb6/volumes/kubernetes.io~projected/kube-api-access-rmnb5 major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5bb1115f-702d-4b53-8852-4b38e452edb6/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/5bb1115f-702d-4b53-8852-4b38e452edb6/volumes/kubernetes.io~secret/metrics-tls major:0 minor:231 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6c4e843c-e392-42ed-b842-508a67b0f64b/volumes/kubernetes.io~projected/kube-api-access-xftgr:{mountpoint:/var/lib/kubelet/pods/6c4e843c-e392-42ed-b842-508a67b0f64b/volumes/kubernetes.io~projected/kube-api-access-xftgr major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7810845b-042a-4fe6-bc58-cf7285f7e396/volumes/kubernetes.io~projected/kube-api-access-5jpd5:{mountpoint:/var/lib/kubelet/pods/7810845b-042a-4fe6-bc58-cf7285f7e396/volumes/kubernetes.io~projected/kube-api-access-5jpd5 major:0 minor:344 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7810845b-042a-4fe6-bc58-cf7285f7e396/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/7810845b-042a-4fe6-bc58-cf7285f7e396/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:340 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7810845b-042a-4fe6-bc58-cf7285f7e396/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/7810845b-042a-4fe6-bc58-cf7285f7e396/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:343 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050/volumes/kubernetes.io~projected/kube-api-access-mpqzl:{mountpoint:/var/lib/kubelet/pods/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050/volumes/kubernetes.io~projected/kube-api-access-mpqzl major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/82e24a0f-8277-484a-b7b4-50dcfb2b4fdd/volumes/kubernetes.io~projected/kube-api-access-zvp55:{mountpoint:/var/lib/kubelet/pods/82e24a0f-8277-484a-b7b4-50dcfb2b4fdd/volumes/kubernetes.io~projected/kube-api-access-zvp55 major:0 minor:279 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/83db651d-42a9-48ba-a7ef-263e32d237d9/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/83db651d-42a9-48ba-a7ef-263e32d237d9/volumes/kubernetes.io~secret/agent-certs major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9909a4b9-77f0-4be2-9374-26a602a72987/volumes/kubernetes.io~projected/kube-api-access-cq2fx:{mountpoint:/var/lib/kubelet/pods/9909a4b9-77f0-4be2-9374-26a602a72987/volumes/kubernetes.io~projected/kube-api-access-cq2fx major:0 minor:345 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9909a4b9-77f0-4be2-9374-26a602a72987/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/9909a4b9-77f0-4be2-9374-26a602a72987/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:342 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9909a4b9-77f0-4be2-9374-26a602a72987/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/9909a4b9-77f0-4be2-9374-26a602a72987/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:341 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/99a606bd-b4a1-43e9-8525-a7afcd1eae0f/volumes/kubernetes.io~projected/kube-api-access-vklnx:{mountpoint:/var/lib/kubelet/pods/99a606bd-b4a1-43e9-8525-a7afcd1eae0f/volumes/kubernetes.io~projected/kube-api-access-vklnx major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9baf3dfd-f665-42ed-8cad-5f4571bf7194/volumes/kubernetes.io~projected/kube-api-access-h89x7:{mountpoint:/var/lib/kubelet/pods/9baf3dfd-f665-42ed-8cad-5f4571bf7194/volumes/kubernetes.io~projected/kube-api-access-h89x7 major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9baf3dfd-f665-42ed-8cad-5f4571bf7194/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/9baf3dfd-f665-42ed-8cad-5f4571bf7194/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:238 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9df1fb81-e74f-45cb-b07a-026f323567b5/volumes/kubernetes.io~projected/kube-api-access-8hwxg:{mountpoint:/var/lib/kubelet/pods/9df1fb81-e74f-45cb-b07a-026f323567b5/volumes/kubernetes.io~projected/kube-api-access-8hwxg major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a1136554-e895-435d-a306-02744ddefdb2/volumes/kubernetes.io~projected/kube-api-access-vqkfd:{mountpoint:/var/lib/kubelet/pods/a1136554-e895-435d-a306-02744ddefdb2/volumes/kubernetes.io~projected/kube-api-access-vqkfd major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a1136554-e895-435d-a306-02744ddefdb2/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/a1136554-e895-435d-a306-02744ddefdb2/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ac2e1d23-6505-4936-9c05-70a76f8fd886/volumes/kubernetes.io~projected/kube-api-access-f5tw5:{mountpoint:/var/lib/kubelet/pods/ac2e1d23-6505-4936-9c05-70a76f8fd886/volumes/kubernetes.io~projected/kube-api-access-f5tw5 major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b0e60cf0-1764-4c18-ab4e-da874feaff36/volumes/kubernetes.io~projected/kube-api-access-4nptl:{mountpoint:/var/lib/kubelet/pods/b0e60cf0-1764-4c18-ab4e-da874feaff36/volumes/kubernetes.io~projected/kube-api-access-4nptl major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b0e60cf0-1764-4c18-ab4e-da874feaff36/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/b0e60cf0-1764-4c18-ab4e-da874feaff36/volumes/kubernetes.io~secret/metrics-certs major:0 minor:304 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bf3a5689-535b-464f-9668-8d389d6396c5/volumes/kubernetes.io~projected/kube-api-access-4lhhb:{mountpoint:/var/lib/kubelet/pods/bf3a5689-535b-464f-9668-8d389d6396c5/volumes/kubernetes.io~projected/kube-api-access-4lhhb major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d7741ae3-0846-4449-a738-45251ad287df/volumes/kubernetes.io~secret/tls-certificates:{mountpoint:/var/lib/kubelet/pods/d7741ae3-0846-4449-a738-45251ad287df/volumes/kubernetes.io~secret/tls-certificates major:0 minor:269 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e1fc5a00-d482-4aa6-955d-ce6ff425240e/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/e1fc5a00-d482-4aa6-955d-ce6ff425240e/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e1fc5a00-d482-4aa6-955d-ce6ff425240e/volumes/kubernetes.io~projected/kube-api-access-rsqms:{mountpoint:/var/lib/kubelet/pods/e1fc5a00-d482-4aa6-955d-ce6ff425240e/volumes/kubernetes.io~projected/kube-api-access-rsqms major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e1fc5a00-d482-4aa6-955d-ce6ff425240e/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/e1fc5a00-d482-4aa6-955d-ce6ff425240e/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:56 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/45e7631d4952cbe42e6e3aa7a4fcaffd4f130fdf12037c47d75631942372398d/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/9f2296861776ead1e7e4410fce6e546a7239903e3cc8275a2ec74c6d885ac328/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/42c3039af05340b76f913cd48d8e0787ed52b6ebda78aa40e551be4b4eca6a11/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/1fb4f463f416fa364d0fb331989bbde6cb4c10e68a06a1b79ce3d96c6dd1ca54/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/4394261d728eeb4f8732d911e2e847c4d1a9f28e55375ee702f7ce2e7bb364d4/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/4519a623c2d077724250c8e9a5424ffea5de21032cc1a5e3de5430c6a15edbc5/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/8f22d43bc299d79ea6f2112f9427b9a0c2f3c44425fa46dbef242cd8f8975ed9/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/b0403cc710ebc3368c0d1c61b922bf50d6351b44e368f0764a1c8672680372a7/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-128:{mountpoint:/var/lib/containers/storage/overlay/bb292ce5600fade2e888716e2d84392b5e2d4e5b366ca1807bc8832f335e1ff1/merged major:0 minor:128 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/120233e46af316abb24d072e72045fe7ed84310834f254556df7b82a548e2f3b/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-136:{mountpoint:/var/lib/containers/storage/overlay/c4f57d0869282d69da7339c2ee46a24e4a11d65703258ac13c620b970a67aaa0/merged major:0 minor:136 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/5962c7ddafbc64ee70d44a349a4f202358bfb733323042f77ff160762e03c1b5/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/6fd6f7bd02357674ac8679f5389d1e7c6cd62cc4d7a3b9b9d7e00a5c5b53c097/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/2f3ee286bc3980b17a092eda80b75c69e8b808643519a4bcc6e6bf33c888c942/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/d7ccbbc8faf0179579efb8e5352c27ba33ef4178d5235e4d009667afb64259ac/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/489e2f3b8fbcfc4597b2818208f550f2bcbeaf4be7b68a0992f4d9eed59dfdeb/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/9b84c330583f8705a768f7f57d53d737edc94a6cc7bd8e70a5c2fbb1ea39760f/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/e9a3bec46a450af07a0da531adf093192a6e3dfe344778f2c42d6dde0a62d45b/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/bfdb0ec23a55170b5adb7ab541b9085e52f055d8e81a5ffbd8a4efc972223b89/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/4bbe9818e2829673176d538eedccb912579ef25cf21e582584190f0a8bb47bf2/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/26683c45cba9d40a015c89b43bf4011c1d6a92edecf4c07a486535be850fd3b4/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-205:{mountpoint:/var/lib/containers/storage/overlay/25c471a22b9cc899c5b0a03dd4e9e5e809a88adc54c7e48c9acc60b4bde05a04/merged major:0 minor:205 fsType:overlay blockSize:0} overlay_0-210:{mountpoint:/var/lib/containers/storage/overlay/dba8ec6bd9ae757aa00af9bff78a3e661cf35ca4fb46923c8ae7d28016b75259/merged major:0 minor:210 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/5ce2a1c176bdcd331d44e12dbfa475f40bcce4fc8d3c01c47489f14c8f84757e/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/f1f110eaf38ba8407b38a18e488bec2817accce338dce676c1fe02703203202c/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/ea24c43db52f2d34a8c127b49bd42f1ab2e8f98b9a1d9e22dcf6fa1b3656fb6b/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/3baf9048b93e86efabe450b1b8e115745c140c1daf1858c4df578ee294fd8a34/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-239:{mountpoint:/var/lib/containers/storage/overlay/58b5ee44ef465f40e2665b28bf0815b7118f737bcb9fa9cbd64ea00c33feed22/merged major:0 minor:239 fsType:overlay blockSize:0} overlay_0-243:{mountpoint:/var/lib/containers/storage/overlay/b52450ce96453de369809f6c016faca565700220f9a8655f8863d9bb9d215e9f/merged major:0 minor:243 fsType:overlay blockSize:0} overlay_0-245:{mountpoint:/var/lib/containers/storage/overlay/0a6eeb925df4f3f4753a68665d81e3a072297d2e9dd490486dda5ea05d60dda8/merged major:0 minor:245 fsType:overlay blockSize:0} overlay_0-247:{mountpoint:/var/lib/containers/storage/overlay/ae9850526415c04bc3c7e437eb70fcf77015ce1bca5854d5f3ac22bc66135096/merged major:0 minor:247 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/d7e641dc4b80ee80b7d9d63555222e67e5f8adbd09b37d882f5230309907e437/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/bfdb9a72ea9077888bb0a53bab7d5d506f3cc766a8510085cf5f54c46f0f5f32/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/60413479d575c4762c6fb750f2a618904afbbfdf0303589a9067187b58b136ca/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-285:{mountpoint:/var/lib/containers/storage/overlay/8283266d079232f082097cb2e05a6886300eed161914907ea37e25ec77e3e776/merged major:0 minor:285 fsType:overlay blockSize:0} overlay_0-287:{mountpoint:/var/lib/containers/storage/overlay/160b60e1c365057d0629c0d5f66af3cfa89bc6ec4ab9924fb122bac897b4c672/merged major:0 minor:287 fsType:overlay blockSize:0} overlay_0-289:{mountpoint:/var/lib/containers/storage/overlay/4c64488760f0033ac677036a2104ad96014c636110886e10fe149b249bca7a4c/merged major:0 minor:289 fsType:overlay blockSize:0} overlay_0-291:{mountpoint:/var/lib/containers/storage/overlay/99807c8c53443b3829c82ee791e5c1c5b49efc94e79d2af2db8666dd02c43a5a/merged major:0 minor:291 fsType:overlay blockSize:0} overlay_0-293:{mountpoint:/var/lib/containers/storage/overlay/5b0ed15514dfb2bc180975f06254731f8e7d6e84f5fa2625d0c4f1ca38b29c81/merged major:0 minor:293 fsType:overlay blockSize:0} overlay_0-312:{mountpoint:/var/lib/containers/storage/overlay/9bf88a5c8ab616634f70924abaf421404eaf577433b3c5980675a4c73012a30e/merged major:0 minor:312 fsType:overlay blockSize:0} overlay_0-314:{mountpoint:/var/lib/containers/storage/overlay/8d84e9294d0aa68b11c04e5624bc5203fbd4586a7ed116664c0fe0549f51fd5f/merged major:0 minor:314 fsType:overlay blockSize:0} overlay_0-316:{mountpoint:/var/lib/containers/storage/overlay/bbf5847d4300772608c3cb7e18d050bd9aeff5e5b8cb6a8ef0464ec7738789a8/merged major:0 minor:316 fsType:overlay blockSize:0} overlay_0-318:{mountpoint:/var/lib/containers/storage/overlay/c3e458258f2a6679a0b9b90b8b80428520ae7b25f4563a482b5749237d346dfc/merged major:0 minor:318 fsType:overlay blockSize:0} overlay_0-320:{mountpoint:/var/lib/containers/storage/overlay/86a3c73727416b67049d3520f7ca7c757aba8264808f6c98df0f3b5b7d79ae9c/merged major:0 minor:320 fsType:overlay blockSize:0} overlay_0-326:{mountpoint:/var/lib/containers/storage/overlay/7ac887e81d9d7393730b1f88847220887cfb4ccb57aa87b223e1a9dd807c1fe6/merged major:0 minor:326 fsType:overlay blockSize:0} overlay_0-350:{mountpoint:/var/lib/containers/storage/overlay/3f43c9854dc9a5290823e6263f2a5c10b21353cb5f8cc7c5af1ed0f2b207b843/merged major:0 minor:350 fsType:overlay blockSize:0} overlay_0-352:{mountpoint:/var/lib/containers/storage/overlay/8965c117ee743c4ee0837a473723c0a4b920eb021da6b17516056f500b4f6b33/merged major:0 minor:352 fsType:overlay blockSize:0} overlay_0-354:{mountpoint:/var/lib/containers/storage/overlay/fc8428f9a40bdb8e1ab998946be3b7b03141726a32efd44a91ab77dc3fe6cbfe/merged major:0 minor:354 fsType:overlay blockSize:0} overlay_0-356:{mountpoint:/var/lib/containers/storage/overlay/c509da38888a18d463541b38d71679ad82be32049a4aa3d7cecbd8b317d6519f/merged major:0 minor:356 fsType:overlay blockSize:0} overlay_0-362:{mountpoint:/var/lib/containers/storage/overlay/a4aa28910ef96e4e8ee2f88d98fc7445f689fa67e92e7f6e06386f3ed0035284/merged major:0 minor:362 fsType:overlay blockSize:0} overlay_0-372:{mountpoint:/var/lib/containers/storage/overlay/ce10e0d2c05a04e110fa06279e2286516911931a46b492317e0be8654e39b34e/merged major:0 minor:372 fsType:overlay blockSize:0} overlay_0-374:{mountpoint:/var/lib/containers/storage/overlay/fd2f2bec1fd73e63e590cf5cba13a6d8d6bc87fb0b3e611b689ce5b44b03a76a/merged major:0 minor:374 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/e1790b951f847d98d629c544cb3f4e17baa244100e1498ff2ea5dee44bd0da68/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/be0c8d42bdfdb56144a4974a851db1e2bdfbd3c1c910733f4804e85f34fa46a4/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/82867d8aa5720262c7cab826a1c39bb283c3c8c8c12138f09f21e1045b0e8c81/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/010ed8dc4a8770eb0c0155e18846d013f9c285466eb6f90297190b82af552002/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/fb2bb4225ffcff8254030a25ffb1c519f01ce83dac619e12d8ebea6ad73ca73c/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/d8371847512ef3c906860c741c08c02ffa7729d2e1d2bc1cf8ab38da390fff9f/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/4703409b0836e794db92d6be03d0b6da83c8b34ca16e1b3ddfa6b7272ce04c21/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/17660bb95549a12090037f6bb9f0445c760800e1273d45b884fb5e3771d3c473/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/fbee1230d636a58e7987621b4f47a6c0ee29b34b6070f71fc12ce7cf6e86877b/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/b4781db87c6c435bd6e0eed400e96ec5505a99ee8eaa5037e2b5fd00a0a220cf/merged major:0 minor:96 fsType:overlay blockSize:0}] Nov 04 05:27:46.208625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.203613 5258 manager.go:217] Machine: {Timestamp:2026-11-04 05:27:46.202895361 +0000 UTC m=+0.082287829 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099579 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2037d248ddaebca97eee23effbd6de SystemUUID:ec2037d2-48dd-aebc-a97e-ee23effbd6de BootID:65c2727b-fa96-4e73-820a-4fd3a022ab2c Filesystems:[{Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a0e86e8bd29d3b22725f3b36e1864b50e3cd003841456838da9511ec22ba3da5/userdata/shm DeviceMajor:0 DeviceMinor:308 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9df1fb81-e74f-45cb-b07a-026f323567b5/volumes/kubernetes.io~projected/kube-api-access-8hwxg DeviceMajor:0 DeviceMinor:61 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-128 DeviceMajor:0 DeviceMinor:128 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5bb1115f-702d-4b53-8852-4b38e452edb6/volumes/kubernetes.io~projected/kube-api-access-rmnb5 DeviceMajor:0 DeviceMinor:197 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/00c77006-a012-4f0e-a886-69ff0150de6d/volumes/kubernetes.io~projected/kube-api-access-6b7cc DeviceMajor:0 DeviceMinor:212 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4806f73f-d8d3-41a3-865f-9df7fe88e1b8/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:277 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4dcc3e21-060d-4d88-a034-025ee61af3ff/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:331 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e456f1812cccb3da24cd453c17ed3fc2f10fe153298688919214022cc2035c00/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-136 DeviceMajor:0 DeviceMinor:136 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/09f4f8137bae50dd458fac2cf6749649224d80f65839a31044896d947a39ed15/userdata/shm DeviceMajor:0 DeviceMinor:306 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4806f73f-d8d3-41a3-865f-9df7fe88e1b8/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:278 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b6c2dbf7b1476ea572b7848a417ec1412e9c6f2704f5e77696912d547c9e8c3e/userdata/shm DeviceMajor:0 DeviceMinor:281 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-312 DeviceMajor:0 DeviceMinor:312 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9909a4b9-77f0-4be2-9374-26a602a72987/volumes/kubernetes.io~projected/kube-api-access-cq2fx DeviceMajor:0 DeviceMinor:345 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9baf3dfd-f665-42ed-8cad-5f4571bf7194/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:238 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-287 DeviceMajor:0 DeviceMinor:287 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-354 DeviceMajor:0 DeviceMinor:354 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1e1ee7a2fb796112615a893a9293f3886537d412fe3873c3445751d9253ff591/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-247 DeviceMajor:0 DeviceMinor:247 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4806f73f-d8d3-41a3-865f-9df7fe88e1b8/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:275 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/23cf3ade-4fac-44f1-813c-7f61427e2b83/volumes/kubernetes.io~projected/kube-api-access-r2bc5 DeviceMajor:0 DeviceMinor:305 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a1136554-e895-435d-a306-02744ddefdb2/volumes/kubernetes.io~projected/kube-api-access-vqkfd DeviceMajor:0 DeviceMinor:201 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b09efdaca27be91789cf702b91cff41442e4c9e4b102ee2c6463478c3f82e0b7/userdata/shm DeviceMajor:0 DeviceMinor:241 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4dcc3e21-060d-4d88-a034-025ee61af3ff/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:336 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-352 DeviceMajor:0 DeviceMinor:352 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-356 DeviceMajor:0 DeviceMinor:356 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-316 DeviceMajor:0 DeviceMinor:316 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-374 DeviceMajor:0 DeviceMinor:374 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/83db651d-42a9-48ba-a7ef-263e32d237d9/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:55 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4fc979bb-e663-4465-a2cc-16cb06be4963/volumes/kubernetes.io~projected/kube-api-access-nzngz DeviceMajor:0 DeviceMinor:67 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-245 DeviceMajor:0 DeviceMinor:245 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a1136554-e895-435d-a306-02744ddefdb2/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4806f73f-d8d3-41a3-865f-9df7fe88e1b8/volumes/kubernetes.io~projected/kube-api-access-2tcm2 DeviceMajor:0 DeviceMinor:280 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7810845b-042a-4fe6-bc58-cf7285f7e396/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:340 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4fc979bb-e663-4465-a2cc-16cb06be4963/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:60 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b0e60cf0-1764-4c18-ab4e-da874feaff36/volumes/kubernetes.io~projected/kube-api-access-4nptl DeviceMajor:0 DeviceMinor:64 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-243 DeviceMajor:0 DeviceMinor:243 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5a710da8-8925-4e16-b43d-fc06a999569b/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:299 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-318 DeviceMajor:0 DeviceMinor:318 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a2cc34cd793b26d5679286cb22c1afd256a95096c2c83af5b04a0dc51c24bdcf/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d7741ae3-0846-4449-a738-45251ad287df/volumes/kubernetes.io~secret/tls-certificates DeviceMajor:0 DeviceMinor:269 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9baf3dfd-f665-42ed-8cad-5f4571bf7194/volumes/kubernetes.io~projected/kube-api-access-h89x7 DeviceMajor:0 DeviceMinor:204 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/5bb1115f-702d-4b53-8852-4b38e452edb6/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:231 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bf3a5689-535b-464f-9668-8d389d6396c5/volumes/kubernetes.io~projected/kube-api-access-4lhhb DeviceMajor:0 DeviceMinor:66 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7dac931844b45988d66af94c90834d70aa8f08c60d881ddb099b21a1b925e5a0/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4806f73f-d8d3-41a3-865f-9df7fe88e1b8/volumes/kubernetes.io~projected/registry-tls DeviceMajor:0 DeviceMinor:276 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-291 DeviceMajor:0 DeviceMinor:291 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b0e60cf0-1764-4c18-ab4e-da874feaff36/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:304 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/46d6e055c92ced70e25d887a457d36afe3656f32caae299a832604f3d72f46cb/userdata/shm DeviceMajor:0 DeviceMinor:338 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-210 DeviceMajor:0 DeviceMinor:210 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-239 DeviceMajor:0 DeviceMinor:239 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/82e24a0f-8277-484a-b7b4-50dcfb2b4fdd/volumes/kubernetes.io~projected/kube-api-access-zvp55 DeviceMajor:0 DeviceMinor:279 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1fa2fef846d263f0e570b3c8a9f936acd1a5f014e1a7f3ada3b50c735aa4d8f5/userdata/shm DeviceMajor:0 DeviceMinor:310 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9909a4b9-77f0-4be2-9374-26a602a72987/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:342 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/e1fc5a00-d482-4aa6-955d-ce6ff425240e/volumes/kubernetes.io~projected/kube-api-access-rsqms DeviceMajor:0 DeviceMinor:57 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e06cf59f7386c12d3abbe5721afac2d9bb4ec36682b2747705eb514275347b6d/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/32c7911aad73a69827aa71e308653a4a1881fca1a866a239e682ee0fb082b3b8/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fbaa1a138a1355400c58c2fbd44b7cd69611c765efe58c7e2ae239c8086b05c6/userdata/shm DeviceMajor:0 DeviceMinor:208 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-293 DeviceMajor:0 DeviceMinor:293 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4dcc3e21-060d-4d88-a034-025ee61af3ff/volumes/kubernetes.io~projected/kube-api-access-tvqg7 DeviceMajor:0 DeviceMinor:337 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/77b95a272239a218daaf92e07e09e4b57be0328dc6885234f7911f53112e8631/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050/volumes/kubernetes.io~projected/kube-api-access-mpqzl DeviceMajor:0 DeviceMinor:63 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e1fc5a00-d482-4aa6-955d-ce6ff425240e/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4ef62602015af169a11d55bbd552150ac8a077a1aa4ca2fa69349c9de0faa96d/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-326 DeviceMajor:0 DeviceMinor:326 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9909a4b9-77f0-4be2-9374-26a602a72987/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:341 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7810845b-042a-4fe6-bc58-cf7285f7e396/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:343 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/99a606bd-b4a1-43e9-8525-a7afcd1eae0f/volumes/kubernetes.io~projected/kube-api-access-vklnx DeviceMajor:0 DeviceMinor:65 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/24cdb6fe97a2013caf78391931390bf69ab3ce4a9eaba0a900ce8608f2503d69/userdata/shm DeviceMajor:0 DeviceMinor:93 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2a9d4ef75ba4781c64c7c7c60ddf779f1b146f6a366f618c4b12bdd284719c10/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4a120e88-95d0-4176-ab05-586d4c2e7dab/volumes/kubernetes.io~projected/kube-api-access-bll88 DeviceMajor:0 DeviceMinor:207 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-285 DeviceMajor:0 DeviceMinor:285 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/11b27b07-7ada-441c-8799-e3d3d111bc18/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:368 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-205 DeviceMajor:0 DeviceMinor:205 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c1b7a155d8e593e9cc5948587dcc48347fc8335dbd52a99eb517fbccf3cef547/userdata/shm DeviceMajor:0 DeviceMinor:213 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9d880bbb709757092439f940aec40765b05af6049d4904cba4b6734b167190a0/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f24431ce12e2fb36139b8e783f12f8402f9c33ee32afe71c6d090c111a9bb5b2/userdata/shm DeviceMajor:0 DeviceMinor:283 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7810845b-042a-4fe6-bc58-cf7285f7e396/volumes/kubernetes.io~projected/kube-api-access-5jpd5 DeviceMajor:0 DeviceMinor:344 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-350 DeviceMajor:0 DeviceMinor:350 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/11b27b07-7ada-441c-8799-e3d3d111bc18/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:364 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6c4e843c-e392-42ed-b842-508a67b0f64b/volumes/kubernetes.io~projected/kube-api-access-xftgr DeviceMajor:0 DeviceMinor:58 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7dd94360c70bdb8282c2497a257293d5cc50c73f8a0c7335f4f731319dac7186/userdata/shm DeviceMajor:0 DeviceMinor:81 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/11b27b07-7ada-441c-8799-e3d3d111bc18/volumes/kubernetes.io~projected/kube-api-access-s58tj DeviceMajor:0 DeviceMinor:369 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-372 DeviceMajor:0 DeviceMinor:372 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ac2e1d23-6505-4936-9c05-70a76f8fd886/volumes/kubernetes.io~projected/kube-api-access-f5tw5 DeviceMajor:0 DeviceMinor:62 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-289 DeviceMajor:0 DeviceMinor:289 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-320 DeviceMajor:0 DeviceMinor:320 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d353d62f4bcc28a17a12b87e478a93b66ce4b99738e4bd8066eef5b9ff8ddfc2/userdata/shm DeviceMajor:0 DeviceMinor:348 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-362 DeviceMajor:0 DeviceMinor:362 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e1fc5a00-d482-4aa6-955d-ce6ff425240e/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:56 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0a960f30f55e9ac08398126969050a8c92023b2d69187c0d0eea1a543fea9c06/userdata/shm DeviceMajor:0 DeviceMinor:273 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/44e9081016a2d01003ff747c050f2c3c9f81da9cc01e11bbf8f204409360c56d/userdata/shm DeviceMajor:0 DeviceMinor:346 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7af0d5e931332006d648b413e3d1c89f456a6b0dae1ab2e542ebad9416d483e4/userdata/shm DeviceMajor:0 DeviceMinor:370 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4fc979bb-e663-4465-a2cc-16cb06be4963/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:59 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-314 DeviceMajor:0 DeviceMinor:314 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:09f4f8137bae50d MacAddress:be:f6:73:53:35:52 Speed:10000 Mtu:8901} {Name:0a960f30f55e9ac MacAddress:d6:3b:53:25:bf:eb Speed:10000 Mtu:8901} {Name:1fa2fef846d263f MacAddress:12:ed:29:1e:16:3e Speed:10000 Mtu:8901} {Name:44e9081016a2d01 MacAddress:06:63:46:5e:ec:95 Speed:10000 Mtu:8901} {Name:46d6e055c92ced7 MacAddress:8a:be:a5:89:d7:42 Speed:10000 Mtu:8901} {Name:4ef62602015af16 MacAddress:d6:76:76:82:7f:f3 Speed:10000 Mtu:8901} {Name:7af0d5e93133200 MacAddress:9e:51:69:d0:42:ae Speed:10000 Mtu:8901} {Name:a0e86e8bd29d3b2 MacAddress:d2:6e:15:59:19:1e Speed:10000 Mtu:8901} {Name:a2cc34cd793b26d MacAddress:46:ca:fe:a2:de:09 Speed:10000 Mtu:8901} {Name:b09efdaca27be91 MacAddress:f6:c7:cc:fd:39:27 Speed:10000 Mtu:8901} {Name:b6c2dbf7b1476ea MacAddress:6e:3c:2f:ef:d4:40 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:fe:41:f2:0b Speed:0 Mtu:9001} {Name:br-int MacAddress:b6:b6:56:12:17:aa Speed:0 Mtu:8901} {Name:c1b7a155d8e593e MacAddress:de:8e:ff:33:ee:a0 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:06:ff:fe:41:f2:0b Speed:0 Mtu:9001} {Name:f24431ce12e2fb3 MacAddress:ba:65:ec:ae:0d:e4 Speed:10000 Mtu:8901} {Name:fbaa1a138a13554 MacAddress:76:7d:e8:46:b3:17 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:22:f2:79:4c:15:ca Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:b2:2b:a1:a0:71:eb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Nov 04 05:27:46.208625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.204149 5258 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Nov 04 05:27:46.208625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.204212 5258 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Nov 04 05:27:46.208625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.204533 5258 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Nov 04 05:27:46.208625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.204555 5258 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-185.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Nov 04 05:27:46.208625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.204749 5258 topology_manager.go:143] "Creating topology manager with none policy" Nov 04 05:27:46.208625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.204757 5258 container_manager_linux.go:311] "Creating device plugin manager" Nov 04 05:27:46.208625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.204776 5258 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 05:27:46.208625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.204793 5258 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 05:27:46.208625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.204897 5258 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Nov 04 05:27:46.208625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.205139 5258 state_mem.go:41] "Initialized" logger="CPUManager state memory" Nov 04 05:27:46.208625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.205252 5258 server.go:1235] "Using root directory" path="/var/lib/kubelet" Nov 04 05:27:46.208625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.205287 5258 kubelet.go:493] "Attempting to sync node with API server" Nov 04 05:27:46.208625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.205310 5258 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Nov 04 05:27:46.208625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.205324 5258 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Nov 04 05:27:46.208625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.205359 5258 kubelet.go:405] "Adding apiserver pod source" Nov 04 05:27:46.208625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.205370 5258 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Nov 04 05:27:46.208625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.206245 5258 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Nov 04 05:27:46.208625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.206258 5258 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 05:27:46.208625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.208187 5258 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Nov 04 05:27:46.221503 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.221482 5258 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Nov 04 05:27:46.221503 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.221499 5258 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 05:27:46.221661 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.221610 5258 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Nov 04 05:27:46.221901 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.221880 5258 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Nov 04 05:27:46.222021 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.221923 5258 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Nov 04 05:27:46.222123 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.222057 5258 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Nov 04 05:27:46.222123 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.222076 5258 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Nov 04 05:27:46.222123 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.222085 5258 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Nov 04 05:27:46.222123 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.222093 5258 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Nov 04 05:27:46.222123 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.222099 5258 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Nov 04 05:27:46.222123 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.222105 5258 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Nov 04 05:27:46.222123 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.222110 5258 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Nov 04 05:27:46.222123 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.222116 5258 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Nov 04 05:27:46.222123 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.222125 5258 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Nov 04 05:27:46.222483 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.222134 5258 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Nov 04 05:27:46.222483 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.222144 5258 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Nov 04 05:27:46.222483 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.222158 5258 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Nov 04 05:27:46.222483 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.222185 5258 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Nov 04 05:27:46.222483 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.222193 5258 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Nov 04 05:27:46.225922 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.225902 5258 server.go:1263] "Started kubelet" Nov 04 05:27:46.226083 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.225986 5258 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Nov 04 05:27:46.226167 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.226108 5258 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Nov 04 05:27:46.226228 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.226195 5258 server_v1.go:49] "podresources" method="list" useActivePods=true Nov 04 05:27:46.226528 ip-10-0-130-185 systemd[1]: Started Kubernetes Kubelet. Nov 04 05:27:46.227692 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.226678 5258 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Nov 04 05:27:46.227692 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.227367 5258 server.go:317] "Adding debug handlers to kubelet server" Nov 04 05:27:46.228867 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.228853 5258 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Nov 04 05:27:46.228955 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.228928 5258 volume_manager.go:309] "The desired_state_of_world populator starts" Nov 04 05:27:46.228955 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.228944 5258 volume_manager.go:311] "Starting Kubelet Volume Manager" Nov 04 05:27:46.228955 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.228915 5258 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Nov 04 05:27:46.229162 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.228995 5258 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Nov 04 05:27:46.229162 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:27:46.229002 5258 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-185.us-west-2.compute.internal\" not found" Nov 04 05:27:46.229162 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.229019 5258 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 05:21:34 +0000 UTC" deadline="2028-06-30 07:27:04.674354758 +0000 UTC" Nov 04 05:27:46.229162 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.229064 5258 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14497h59m18.445294048s" Nov 04 05:27:46.233096 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.231667 5258 factory.go:55] Registering systemd factory Nov 04 05:27:46.233096 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.231690 5258 factory.go:223] Registration of the systemd container factory successfully Nov 04 05:27:46.233096 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.231946 5258 factory.go:153] Registering CRI-O factory Nov 04 05:27:46.233096 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.231962 5258 factory.go:223] Registration of the crio container factory successfully Nov 04 05:27:46.233096 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.232063 5258 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Nov 04 05:27:46.233096 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.232090 5258 factory.go:103] Registering Raw factory Nov 04 05:27:46.233096 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.232108 5258 manager.go:1199] Started watching for new ooms in manager Nov 04 05:27:46.233096 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.232531 5258 manager.go:319] Starting recovery of all containers Nov 04 05:27:46.233576 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233311 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5a710da8-8925-4e16-b43d-fc06a999569b" volumeName="kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret" seLinuxMountContext="" Nov 04 05:27:46.233576 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233366 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99a606bd-b4a1-43e9-8525-a7afcd1eae0f" volumeName="kubernetes.io/projected/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-kube-api-access-vklnx" seLinuxMountContext="" Nov 04 05:27:46.233576 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233392 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11b27b07-7ada-441c-8799-e3d3d111bc18" volumeName="kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-oauth-serving-cert" seLinuxMountContext="" Nov 04 05:27:46.233576 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233409 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11b27b07-7ada-441c-8799-e3d3d111bc18" volumeName="kubernetes.io/secret/11b27b07-7ada-441c-8799-e3d3d111bc18-console-oauth-config" seLinuxMountContext="" Nov 04 05:27:46.233796 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233580 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="82e24a0f-8277-484a-b7b4-50dcfb2b4fdd" volumeName="kubernetes.io/projected/82e24a0f-8277-484a-b7b4-50dcfb2b4fdd-kube-api-access-zvp55" seLinuxMountContext="" Nov 04 05:27:46.233796 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233595 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d7741ae3-0846-4449-a738-45251ad287df" volumeName="kubernetes.io/secret/d7741ae3-0846-4449-a738-45251ad287df-tls-certificates" seLinuxMountContext="" Nov 04 05:27:46.233796 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233607 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11b27b07-7ada-441c-8799-e3d3d111bc18" volumeName="kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-console-config" seLinuxMountContext="" Nov 04 05:27:46.233796 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233626 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5bb1115f-702d-4b53-8852-4b38e452edb6" volumeName="kubernetes.io/configmap/5bb1115f-702d-4b53-8852-4b38e452edb6-config-volume" seLinuxMountContext="" Nov 04 05:27:46.233796 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233643 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5bb1115f-702d-4b53-8852-4b38e452edb6" volumeName="kubernetes.io/projected/5bb1115f-702d-4b53-8852-4b38e452edb6-kube-api-access-rmnb5" seLinuxMountContext="" Nov 04 05:27:46.233796 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233662 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf3a5689-535b-464f-9668-8d389d6396c5" volumeName="kubernetes.io/empty-dir/bf3a5689-535b-464f-9668-8d389d6396c5-tmp-dir" seLinuxMountContext="" Nov 04 05:27:46.233796 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233677 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7810845b-042a-4fe6-bc58-cf7285f7e396" volumeName="kubernetes.io/projected/7810845b-042a-4fe6-bc58-cf7285f7e396-kube-api-access-5jpd5" seLinuxMountContext="" Nov 04 05:27:46.233796 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233695 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9909a4b9-77f0-4be2-9374-26a602a72987" volumeName="kubernetes.io/empty-dir/9909a4b9-77f0-4be2-9374-26a602a72987-volume-directive-shadow" seLinuxMountContext="" Nov 04 05:27:46.233796 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233708 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9909a4b9-77f0-4be2-9374-26a602a72987" volumeName="kubernetes.io/projected/9909a4b9-77f0-4be2-9374-26a602a72987-kube-api-access-cq2fx" seLinuxMountContext="" Nov 04 05:27:46.233796 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233724 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9baf3dfd-f665-42ed-8cad-5f4571bf7194" volumeName="kubernetes.io/configmap/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-rbac-proxy-cm" seLinuxMountContext="" Nov 04 05:27:46.233796 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233759 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9baf3dfd-f665-42ed-8cad-5f4571bf7194" volumeName="kubernetes.io/empty-dir/9baf3dfd-f665-42ed-8cad-5f4571bf7194-data-volume" seLinuxMountContext="" Nov 04 05:27:46.233796 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233771 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1136554-e895-435d-a306-02744ddefdb2" volumeName="kubernetes.io/secret/a1136554-e895-435d-a306-02744ddefdb2-cert" seLinuxMountContext="" Nov 04 05:27:46.233796 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233794 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4806f73f-d8d3-41a3-865f-9df7fe88e1b8" volumeName="kubernetes.io/projected/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-bound-sa-token" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233807 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4806f73f-d8d3-41a3-865f-9df7fe88e1b8" volumeName="kubernetes.io/secret/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-image-registry-private-configuration" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233826 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6c4e843c-e392-42ed-b842-508a67b0f64b" volumeName="kubernetes.io/projected/6c4e843c-e392-42ed-b842-508a67b0f64b-kube-api-access-xftgr" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233840 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7810845b-042a-4fe6-bc58-cf7285f7e396" volumeName="kubernetes.io/configmap/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-accelerators-collector-config" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233853 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9909a4b9-77f0-4be2-9374-26a602a72987" volumeName="kubernetes.io/configmap/9909a4b9-77f0-4be2-9374-26a602a72987-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233871 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00c77006-a012-4f0e-a886-69ff0150de6d" volumeName="kubernetes.io/projected/00c77006-a012-4f0e-a886-69ff0150de6d-kube-api-access-6b7cc" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233885 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4fc979bb-e663-4465-a2cc-16cb06be4963" volumeName="kubernetes.io/empty-dir/4fc979bb-e663-4465-a2cc-16cb06be4963-tmp" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233903 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4fc979bb-e663-4465-a2cc-16cb06be4963" volumeName="kubernetes.io/projected/4fc979bb-e663-4465-a2cc-16cb06be4963-kube-api-access-nzngz" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233916 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b0e60cf0-1764-4c18-ab4e-da874feaff36" volumeName="kubernetes.io/projected/b0e60cf0-1764-4c18-ab4e-da874feaff36-kube-api-access-4nptl" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233932 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4806f73f-d8d3-41a3-865f-9df7fe88e1b8" volumeName="kubernetes.io/configmap/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-registry-certificates" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233950 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4806f73f-d8d3-41a3-865f-9df7fe88e1b8" volumeName="kubernetes.io/projected/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-registry-tls" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233963 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7810845b-042a-4fe6-bc58-cf7285f7e396" volumeName="kubernetes.io/secret/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-kube-rbac-proxy-config" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.233991 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9df1fb81-e74f-45cb-b07a-026f323567b5" volumeName="kubernetes.io/projected/9df1fb81-e74f-45cb-b07a-026f323567b5-kube-api-access-8hwxg" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.234010 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf3a5689-535b-464f-9668-8d389d6396c5" volumeName="kubernetes.io/projected/bf3a5689-535b-464f-9668-8d389d6396c5-kube-api-access-4lhhb" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.234025 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e1fc5a00-d482-4aa6-955d-ce6ff425240e" volumeName="kubernetes.io/configmap/e1fc5a00-d482-4aa6-955d-ce6ff425240e-env-overrides" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.234057 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9909a4b9-77f0-4be2-9374-26a602a72987" volumeName="kubernetes.io/secret/9909a4b9-77f0-4be2-9374-26a602a72987-kube-state-metrics-tls" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.234078 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4fc979bb-e663-4465-a2cc-16cb06be4963" volumeName="kubernetes.io/empty-dir/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-tuned" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.234091 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11b27b07-7ada-441c-8799-e3d3d111bc18" volumeName="kubernetes.io/empty-dir/11b27b07-7ada-441c-8799-e3d3d111bc18-tmp" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.234111 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a120e88-95d0-4176-ab05-586d4c2e7dab" volumeName="kubernetes.io/configmap/4a120e88-95d0-4176-ab05-586d4c2e7dab-service-ca-bundle" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.234123 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a120e88-95d0-4176-ab05-586d4c2e7dab" volumeName="kubernetes.io/configmap/4a120e88-95d0-4176-ab05-586d4c2e7dab-trusted-ca-bundle" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.234136 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83db651d-42a9-48ba-a7ef-263e32d237d9" volumeName="kubernetes.io/secret/83db651d-42a9-48ba-a7ef-263e32d237d9-agent-certs" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.234153 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b0e60cf0-1764-4c18-ab4e-da874feaff36" volumeName="kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.234167 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11b27b07-7ada-441c-8799-e3d3d111bc18" volumeName="kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-service-ca" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.234184 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11b27b07-7ada-441c-8799-e3d3d111bc18" volumeName="kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-trusted-ca-bundle" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.234197 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a120e88-95d0-4176-ab05-586d4c2e7dab" volumeName="kubernetes.io/projected/4a120e88-95d0-4176-ab05-586d4c2e7dab-kube-api-access-bll88" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.234210 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7810845b-042a-4fe6-bc58-cf7285f7e396" volumeName="kubernetes.io/configmap/7810845b-042a-4fe6-bc58-cf7285f7e396-metrics-client-ca" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.234294 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7810845b-042a-4fe6-bc58-cf7285f7e396" volumeName="kubernetes.io/secret/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-tls" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.234307 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050" volumeName="kubernetes.io/projected/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-kube-api-access-mpqzl" seLinuxMountContext="" Nov 04 05:27:46.234407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.234327 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac2e1d23-6505-4936-9c05-70a76f8fd886" volumeName="kubernetes.io/projected/ac2e1d23-6505-4936-9c05-70a76f8fd886-kube-api-access-f5tw5" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.234341 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4806f73f-d8d3-41a3-865f-9df7fe88e1b8" volumeName="kubernetes.io/empty-dir/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-ca-trust-extracted" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235254 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a120e88-95d0-4176-ab05-586d4c2e7dab" volumeName="kubernetes.io/empty-dir/4a120e88-95d0-4176-ab05-586d4c2e7dab-archives-path" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235278 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5bb1115f-702d-4b53-8852-4b38e452edb6" volumeName="kubernetes.io/secret/5bb1115f-702d-4b53-8852-4b38e452edb6-metrics-tls" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235355 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7810845b-042a-4fe6-bc58-cf7285f7e396" volumeName="kubernetes.io/empty-dir/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-textfile" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235371 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9df1fb81-e74f-45cb-b07a-026f323567b5" volumeName="kubernetes.io/configmap/9df1fb81-e74f-45cb-b07a-026f323567b5-serviceca" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235390 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4dcc3e21-060d-4d88-a034-025ee61af3ff" volumeName="kubernetes.io/configmap/4dcc3e21-060d-4d88-a034-025ee61af3ff-metrics-client-ca" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235404 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9909a4b9-77f0-4be2-9374-26a602a72987" volumeName="kubernetes.io/configmap/9909a4b9-77f0-4be2-9374-26a602a72987-metrics-client-ca" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235420 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11b27b07-7ada-441c-8799-e3d3d111bc18" volumeName="kubernetes.io/projected/11b27b07-7ada-441c-8799-e3d3d111bc18-kube-api-access-s58tj" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235461 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4dcc3e21-060d-4d88-a034-025ee61af3ff" volumeName="kubernetes.io/secret/4dcc3e21-060d-4d88-a034-025ee61af3ff-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235475 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="83db651d-42a9-48ba-a7ef-263e32d237d9" volumeName="kubernetes.io/configmap/83db651d-42a9-48ba-a7ef-263e32d237d9-konnectivity-ca" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235492 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99a606bd-b4a1-43e9-8525-a7afcd1eae0f" volumeName="kubernetes.io/configmap/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-cni-binary-copy" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235531 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99a606bd-b4a1-43e9-8525-a7afcd1eae0f" volumeName="kubernetes.io/configmap/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-cni-sysctl-allowlist" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235545 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1136554-e895-435d-a306-02744ddefdb2" volumeName="kubernetes.io/projected/a1136554-e895-435d-a306-02744ddefdb2-kube-api-access-vqkfd" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235564 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e1fc5a00-d482-4aa6-955d-ce6ff425240e" volumeName="kubernetes.io/configmap/e1fc5a00-d482-4aa6-955d-ce6ff425240e-ovnkube-config" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235579 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e1fc5a00-d482-4aa6-955d-ce6ff425240e" volumeName="kubernetes.io/projected/e1fc5a00-d482-4aa6-955d-ce6ff425240e-kube-api-access-rsqms" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235594 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5bb1115f-702d-4b53-8852-4b38e452edb6" volumeName="kubernetes.io/empty-dir/5bb1115f-702d-4b53-8852-4b38e452edb6-tmp-dir" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235611 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6c4e843c-e392-42ed-b842-508a67b0f64b" volumeName="kubernetes.io/configmap/6c4e843c-e392-42ed-b842-508a67b0f64b-iptables-alerter-script" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235625 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9909a4b9-77f0-4be2-9374-26a602a72987" volumeName="kubernetes.io/secret/9909a4b9-77f0-4be2-9374-26a602a72987-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235746 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="99a606bd-b4a1-43e9-8525-a7afcd1eae0f" volumeName="kubernetes.io/configmap/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-whereabouts-flatfile-configmap" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235765 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9baf3dfd-f665-42ed-8cad-5f4571bf7194" volumeName="kubernetes.io/projected/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-api-access-h89x7" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235778 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac2e1d23-6505-4936-9c05-70a76f8fd886" volumeName="kubernetes.io/configmap/ac2e1d23-6505-4936-9c05-70a76f8fd886-cni-binary-copy" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235797 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23cf3ade-4fac-44f1-813c-7f61427e2b83" volumeName="kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235813 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4806f73f-d8d3-41a3-865f-9df7fe88e1b8" volumeName="kubernetes.io/projected/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-kube-api-access-2tcm2" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235831 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4dcc3e21-060d-4d88-a034-025ee61af3ff" volumeName="kubernetes.io/projected/4dcc3e21-060d-4d88-a034-025ee61af3ff-kube-api-access-tvqg7" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235844 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9baf3dfd-f665-42ed-8cad-5f4571bf7194" volumeName="kubernetes.io/secret/9baf3dfd-f665-42ed-8cad-5f4571bf7194-insights-runtime-extractor-tls" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235858 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e1fc5a00-d482-4aa6-955d-ce6ff425240e" volumeName="kubernetes.io/configmap/e1fc5a00-d482-4aa6-955d-ce6ff425240e-ovnkube-script-lib" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235873 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11b27b07-7ada-441c-8799-e3d3d111bc18" volumeName="kubernetes.io/secret/11b27b07-7ada-441c-8799-e3d3d111bc18-console-serving-cert" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235883 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="82e24a0f-8277-484a-b7b4-50dcfb2b4fdd" volumeName="kubernetes.io/empty-dir/82e24a0f-8277-484a-b7b4-50dcfb2b4fdd-tmp" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235901 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac2e1d23-6505-4936-9c05-70a76f8fd886" volumeName="kubernetes.io/configmap/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-daemon-config" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235914 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4806f73f-d8d3-41a3-865f-9df7fe88e1b8" volumeName="kubernetes.io/configmap/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-trusted-ca" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235928 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4806f73f-d8d3-41a3-865f-9df7fe88e1b8" volumeName="kubernetes.io/secret/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-installation-pull-secrets" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235947 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4dcc3e21-060d-4d88-a034-025ee61af3ff" volumeName="kubernetes.io/secret/4dcc3e21-060d-4d88-a034-025ee61af3ff-openshift-state-metrics-tls" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235960 5258 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e1fc5a00-d482-4aa6-955d-ce6ff425240e" volumeName="kubernetes.io/secret/e1fc5a00-d482-4aa6-955d-ce6ff425240e-ovn-node-metrics-cert" seLinuxMountContext="" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235973 5258 reconstruct.go:97] "Volume reconstruction finished" Nov 04 05:27:46.236407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.235981 5258 reconciler.go:29] "Reconciler: start to sync state" Nov 04 05:27:46.245237 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.245116 5258 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Nov 04 05:27:46.262687 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.262660 5258 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Nov 04 05:27:46.262687 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.262688 5258 status_manager.go:249] "Starting to sync pod status with apiserver" Nov 04 05:27:46.262861 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.262711 5258 kubelet.go:2521] "Starting kubelet main sync loop" Nov 04 05:27:46.262861 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:27:46.262753 5258 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Nov 04 05:27:46.298768 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.298677 5258 generic.go:356] "Generic (PLEG): container finished" podID="99a606bd-b4a1-43e9-8525-a7afcd1eae0f" containerID="a6a0e1cddd62321593a716f1f406b3a4569d3446ef3ff0be8b50c83c8f4b8433" exitCode=0 Nov 04 05:27:46.298768 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.298729 5258 generic.go:356] "Generic (PLEG): container finished" podID="99a606bd-b4a1-43e9-8525-a7afcd1eae0f" containerID="636274521a8199609c88ba5c555e3a63e5ae8b68a8fd8463735dc9e54e3b8671" exitCode=0 Nov 04 05:27:46.298768 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.298745 5258 generic.go:356] "Generic (PLEG): container finished" podID="99a606bd-b4a1-43e9-8525-a7afcd1eae0f" containerID="32d2f9d7fe8426dc8494cc2a13944d887cd31c1189c4e7073693b07abc35e91b" exitCode=0 Nov 04 05:27:46.298768 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.298757 5258 generic.go:356] "Generic (PLEG): container finished" podID="99a606bd-b4a1-43e9-8525-a7afcd1eae0f" containerID="b156e52ba7a890c7101a1737f7be98d60b976087f01c1edba666ff1eee41ba9b" exitCode=0 Nov 04 05:27:46.298768 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.298767 5258 generic.go:356] "Generic (PLEG): container finished" podID="99a606bd-b4a1-43e9-8525-a7afcd1eae0f" containerID="ccdd9f0a1633cc0b5fa9ef34d68e014e1a9aad72f85ed193fa028cbdae0e1dc9" exitCode=0 Nov 04 05:27:46.299120 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.298776 5258 generic.go:356] "Generic (PLEG): container finished" podID="99a606bd-b4a1-43e9-8525-a7afcd1eae0f" containerID="f8b96a7b6326eec6f3ec6c663fef9b72a6f1752322ffafbc324896d894e7e265" exitCode=0 Nov 04 05:27:46.325538 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.325499 5258 generic.go:356] "Generic (PLEG): container finished" podID="dbfe51fb7c8cecce5bf77e4c5e14a1c8" containerID="da6eab7047edd1b7ec50cf1e1c266ef9d31f80a89f6149aca970c0f1ee6d6570" exitCode=0 Nov 04 05:27:46.329521 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:27:46.329500 5258 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-185.us-west-2.compute.internal\" not found" Nov 04 05:27:46.345806 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.345781 5258 manager.go:324] Recovery completed Nov 04 05:27:46.363183 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:27:46.363149 5258 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Nov 04 05:27:46.372535 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.372511 5258 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:27:46.374807 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.374788 5258 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:27:46.374903 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.374826 5258 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:27:46.374903 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.374841 5258 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:27:46.378175 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.378156 5258 cpu_manager.go:226] "Starting" policy="none" Nov 04 05:27:46.378175 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.378172 5258 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Nov 04 05:27:46.378318 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.378193 5258 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Nov 04 05:27:46.378385 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.378361 5258 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Nov 04 05:27:46.378500 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.378383 5258 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Nov 04 05:27:46.378500 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.378408 5258 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Nov 04 05:27:46.378500 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.378416 5258 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Nov 04 05:27:46.378500 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.378425 5258 policy_none.go:50] "Start" Nov 04 05:27:46.378500 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.378437 5258 memory_manager.go:187] "Starting memorymanager" policy="None" Nov 04 05:27:46.378500 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.378451 5258 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Nov 04 05:27:46.378768 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.378638 5258 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Nov 04 05:27:46.378768 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.378650 5258 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Nov 04 05:27:46.378768 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.378658 5258 policy_none.go:44] "Start" Nov 04 05:27:46.378976 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.378803 5258 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:27:46.379718 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.379695 5258 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:27:46.379798 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.379732 5258 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:27:46.379798 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.379747 5258 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:27:46.384156 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.384122 5258 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Nov 04 05:27:46.388630 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.388614 5258 manager.go:341] "Starting Device Plugin manager" Nov 04 05:27:46.388726 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:27:46.388675 5258 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Nov 04 05:27:46.388726 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.388689 5258 server.go:87] "Starting device plugin registration server" Nov 04 05:27:46.388975 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.388962 5258 eviction_manager.go:194] "Eviction manager: starting control loop" Nov 04 05:27:46.389054 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.388974 5258 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Nov 04 05:27:46.389116 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.389095 5258 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Nov 04 05:27:46.389267 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.389219 5258 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Nov 04 05:27:46.389348 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.389278 5258 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Nov 04 05:27:46.389348 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.389293 5258 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Nov 04 05:27:46.389456 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.389357 5258 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T05:27:46.389265239Z","UUID":"cd735361-1930-41cb-b9a8-e2bc995673f4","Handler":null,"Name":"","Endpoint":""} Nov 04 05:27:46.390913 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:27:46.390891 5258 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Nov 04 05:27:46.391016 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:27:46.390935 5258 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-185.us-west-2.compute.internal\" not found" Nov 04 05:27:46.393165 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.393147 5258 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Nov 04 05:27:46.393264 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.393178 5258 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Nov 04 05:27:46.414906 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.414884 5258 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:27:46.415053 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.414974 5258 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Nov 04 05:27:46.415053 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.414988 5258 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:27:46.416140 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.416111 5258 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:27:46.489628 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.489590 5258 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 05:27:46.490800 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.490770 5258 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 05:27:46.490921 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.490811 5258 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 05:27:46.490921 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.490828 5258 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-185.us-west-2.compute.internal" event="NodeHasSufficientPID" Nov 04 05:27:46.490921 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.490889 5258 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:27:46.509503 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.509470 5258 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 05:27:46.564152 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.564058 5258 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal"] Nov 04 05:27:46.564832 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.564808 5258 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:27:46.565067 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.565001 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" event={"ID":"dbfe51fb7c8cecce5bf77e4c5e14a1c8","Type":"ContainerStarted","Data":"71a61d8a6e10b4b328e4875b2c310a169021170d5fee56d3f292c1031cc5bf72"} Nov 04 05:27:46.565149 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.565083 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" event={"ID":"dbfe51fb7c8cecce5bf77e4c5e14a1c8","Type":"ContainerDied","Data":"da6eab7047edd1b7ec50cf1e1c266ef9d31f80a89f6149aca970c0f1ee6d6570"} Nov 04 05:27:46.565149 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.565100 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" event={"ID":"dbfe51fb7c8cecce5bf77e4c5e14a1c8","Type":"ContainerStarted","Data":"1e1ee7a2fb796112615a893a9293f3886537d412fe3873c3445751d9253ff591"} Nov 04 05:27:46.565149 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.565113 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal" event={"ID":"4eff7615010b2db89d82737c9bc2bd5b","Type":"ContainerStarted","Data":"2e9534c89fb0f5d92c2f49d879e10cda28184840ebd40bffa99a0fa3225b7fc4"} Nov 04 05:27:46.565149 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.565124 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal" event={"ID":"4eff7615010b2db89d82737c9bc2bd5b","Type":"ContainerStarted","Data":"77b95a272239a218daaf92e07e09e4b57be0328dc6885234f7911f53112e8631"} Nov 04 05:27:46.565149 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.565133 5258 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:27:46.612281 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.612252 5258 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:27:46.612436 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.612327 5258 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:27:46.623857 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.623833 5258 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:27:46.623986 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:27:46.623887 5258 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:27:46.623986 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.623832 5258 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:27:46.623986 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:27:46.623963 5258 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:27:46.716429 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.716390 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dbfe51fb7c8cecce5bf77e4c5e14a1c8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal\" (UID: \"dbfe51fb7c8cecce5bf77e4c5e14a1c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:27:46.716588 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.716454 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dbfe51fb7c8cecce5bf77e4c5e14a1c8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal\" (UID: \"dbfe51fb7c8cecce5bf77e4c5e14a1c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:27:46.716588 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.716484 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4eff7615010b2db89d82737c9bc2bd5b-config\") pod \"kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal\" (UID: \"4eff7615010b2db89d82737c9bc2bd5b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:27:46.817428 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.817329 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4eff7615010b2db89d82737c9bc2bd5b-config\") pod \"kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal\" (UID: \"4eff7615010b2db89d82737c9bc2bd5b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:27:46.817428 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.817382 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4eff7615010b2db89d82737c9bc2bd5b-config\") pod \"kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal\" (UID: \"4eff7615010b2db89d82737c9bc2bd5b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:27:46.817428 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.817412 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dbfe51fb7c8cecce5bf77e4c5e14a1c8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal\" (UID: \"dbfe51fb7c8cecce5bf77e4c5e14a1c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:27:46.817687 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.817439 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dbfe51fb7c8cecce5bf77e4c5e14a1c8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal\" (UID: \"dbfe51fb7c8cecce5bf77e4c5e14a1c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:27:46.817687 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.817479 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dbfe51fb7c8cecce5bf77e4c5e14a1c8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal\" (UID: \"dbfe51fb7c8cecce5bf77e4c5e14a1c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:27:46.817687 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:46.817484 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/dbfe51fb7c8cecce5bf77e4c5e14a1c8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal\" (UID: \"dbfe51fb7c8cecce5bf77e4c5e14a1c8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:27:47.210983 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.210945 5258 apiserver.go:52] "Watching apiserver" Nov 04 05:27:47.309870 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.309837 5258 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Nov 04 05:27:47.310575 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.310548 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-7b7zs","openshift-monitoring/alertmanager-main-0","openshift-cluster-node-tuning-operator/tuned-dff2s","openshift-image-registry/image-registry-74f7d9b7f7-488q9","openshift-insights/insights-runtime-extractor-qrxks","openshift-multus/multus-additional-cni-plugins-bzbgk","openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pnm9q","openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj","openshift-network-diagnostics/network-check-target-cmc7w","openshift-monitoring/thanos-querier-75d996c778-7l2qj","kube-system/kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z","openshift-console/console-788f8bd759-f6zjh","openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj","openshift-ingress-canary/ingress-canary-g2hdc","openshift-dns/dns-default-8fgr7","openshift-kube-storage-version-migrator/migrator-66df5d496b-hgld4","openshift-monitoring/node-exporter-69gbq","kube-system/konnectivity-agent-z2pjx","openshift-multus/multus-nn7ng","openshift-multus/network-metrics-daemon-chl9q","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal","openshift-ovn-kubernetes/ovnkube-node-cnn7v","openshift-image-registry/node-ca-9bgz9","openshift-insights/periodic-gathering-tfj9k-j9jq9","kube-system/global-pull-secret-syncer-4pvss","openshift-network-operator/iptables-alerter-mbh6p","openshift-console/downloads-6dd789c8b7-f59mk"] Nov 04 05:27:47.328517 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.328490 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.331400 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.331375 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.344726 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.344702 5258 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:27:47.344939 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.344923 5258 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:27:47.362928 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.362896 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Nov 04 05:27:47.362928 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.362895 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Nov 04 05:27:47.362928 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.362896 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:27:47.363217 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.362958 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Nov 04 05:27:47.363217 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.362989 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Nov 04 05:27:47.363217 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.363025 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Nov 04 05:27:47.363217 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.363159 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Nov 04 05:27:47.363217 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.363167 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Nov 04 05:27:47.363217 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.363167 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Nov 04 05:27:47.363217 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.363170 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Nov 04 05:27:47.363217 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.363194 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Nov 04 05:27:47.363513 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.363238 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Nov 04 05:27:47.363513 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.363298 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Nov 04 05:27:47.363596 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.363556 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Nov 04 05:27:47.363646 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.363606 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Nov 04 05:27:47.363646 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.363609 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 05:27:47.363718 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.363643 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Nov 04 05:27:47.363718 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.363664 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 05:27:47.363964 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.363946 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Nov 04 05:27:47.364283 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.364263 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:27:47.364283 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.364278 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Nov 04 05:27:47.364401 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.364278 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Nov 04 05:27:47.364401 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.364345 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Nov 04 05:27:47.368161 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.368143 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Nov 04 05:27:47.369802 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.369788 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Nov 04 05:27:47.369802 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.369792 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Nov 04 05:27:47.370124 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.370107 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Nov 04 05:27:47.391176 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.391150 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Nov 04 05:27:47.391762 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.391738 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vw645\"" Nov 04 05:27:47.391999 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.391969 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cbfrj\"" Nov 04 05:27:47.392127 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.392077 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6nvfw\"" Nov 04 05:27:47.392225 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.392208 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-k2gkz\"" Nov 04 05:27:47.392363 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.392336 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-bnzsz\"" Nov 04 05:27:47.392529 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.392383 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-x7j27\"" Nov 04 05:27:47.392529 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.392493 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-8lkk9\"" Nov 04 05:27:47.392529 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.392507 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-z6mlw\"" Nov 04 05:27:47.392669 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.392545 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 05:27:47.392722 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.392706 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Nov 04 05:27:47.392772 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.392725 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hqws2\"" Nov 04 05:27:47.392827 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.392773 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Nov 04 05:27:47.392882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.392828 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Nov 04 05:27:47.393002 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.392983 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-2psbf\"" Nov 04 05:27:47.393002 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.392991 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r57nx\"" Nov 04 05:27:47.393131 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.393027 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fz9sj\"" Nov 04 05:27:47.395049 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.393198 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qgv9m\"" Nov 04 05:27:47.395049 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.393344 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 05:27:47.407791 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.407771 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-z7zhw\"" Nov 04 05:27:47.407872 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.407851 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tb5s5\"" Nov 04 05:27:47.407953 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.407937 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Nov 04 05:27:47.409062 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.409019 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-gd2dx\"" Nov 04 05:27:47.409168 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.409149 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Nov 04 05:27:47.409228 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.409183 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-svfgt\"" Nov 04 05:27:47.409285 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.409269 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Nov 04 05:27:47.409338 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.409270 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Nov 04 05:27:47.409338 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.409283 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-2ltvz\"" Nov 04 05:27:47.409420 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.409312 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Nov 04 05:27:47.409420 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.409370 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-94czp\"" Nov 04 05:27:47.409511 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.409435 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Nov 04 05:27:47.409564 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.409527 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Nov 04 05:27:47.409597 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.409579 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Nov 04 05:27:47.412840 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.412819 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Nov 04 05:27:47.413080 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.413048 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Nov 04 05:27:47.413223 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.413130 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-sxw5c\"" Nov 04 05:27:47.413283 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.413219 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Nov 04 05:27:47.413283 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.413233 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Nov 04 05:27:47.413283 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.413222 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Nov 04 05:27:47.413623 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.413604 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-tnkgv\"" Nov 04 05:27:47.413623 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.413604 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Nov 04 05:27:47.413762 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.413707 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Nov 04 05:27:47.413762 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.413729 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Nov 04 05:27:47.413762 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.413739 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Nov 04 05:27:47.413762 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.413729 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Nov 04 05:27:47.413924 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.413776 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Nov 04 05:27:47.413924 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.413786 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Nov 04 05:27:47.413924 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.413793 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ptjqz\"" Nov 04 05:27:47.414060 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.413956 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Nov 04 05:27:47.414117 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.414082 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Nov 04 05:27:47.414117 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.414093 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Nov 04 05:27:47.414215 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.414141 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Nov 04 05:27:47.414215 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.414163 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Nov 04 05:27:47.414215 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.414181 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dvjld\"" Nov 04 05:27:47.414215 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.414184 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-9o6h8dd4bq16m\"" Nov 04 05:27:47.414215 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.414199 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Nov 04 05:27:47.414215 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.414208 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Nov 04 05:27:47.414215 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.414098 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-94c2x\"" Nov 04 05:27:47.414215 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.414100 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-szc85\"" Nov 04 05:27:47.414472 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.414227 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6qrg6\"" Nov 04 05:27:47.416378 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.416361 5258 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:27:47.416535 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:27:47.416515 5258 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:27:47.416746 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.416725 5258 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 05:27:47.416818 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:27:47.416771 5258 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-185.us-west-2.compute.internal" Nov 04 05:27:47.419914 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.419895 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvqg7\" (UniqueName: \"kubernetes.io/projected/4dcc3e21-060d-4d88-a034-025ee61af3ff-kube-api-access-tvqg7\") pod \"openshift-state-metrics-7bd7489957-pbhxj\" (UID: \"4dcc3e21-060d-4d88-a034-025ee61af3ff\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" Nov 04 05:27:47.419981 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.419928 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-slash\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.419981 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.419944 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e1fc5a00-d482-4aa6-955d-ce6ff425240e-ovnkube-script-lib\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.419981 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.419960 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqt7m\" (UniqueName: \"kubernetes.io/projected/d3ea1a41-8e39-4d16-ae87-0e030773dc09-kube-api-access-bqt7m\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.420122 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.419984 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-conf-dir\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.420122 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.420022 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a120e88-95d0-4176-ab05-586d4c2e7dab-service-ca-bundle\") pod \"periodic-gathering-tfj9k-j9jq9\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:27:47.420122 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.420107 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4dcc3e21-060d-4d88-a034-025ee61af3ff-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-pbhxj\" (UID: \"4dcc3e21-060d-4d88-a034-025ee61af3ff\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" Nov 04 05:27:47.420259 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.420204 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-systemd\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.420312 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.420284 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e1fc5a00-d482-4aa6-955d-ce6ff425240e-ovnkube-script-lib\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.420362 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.420285 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nptl\" (UniqueName: \"kubernetes.io/projected/b0e60cf0-1764-4c18-ab4e-da874feaff36-kube-api-access-4nptl\") pod \"network-metrics-daemon-chl9q\" (UID: \"b0e60cf0-1764-4c18-ab4e-da874feaff36\") " pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:27:47.420362 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.420348 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-kubernetes\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.420442 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.420377 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-installation-pull-secrets\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:47.420442 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.420384 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4dcc3e21-060d-4d88-a034-025ee61af3ff-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-pbhxj\" (UID: \"4dcc3e21-060d-4d88-a034-025ee61af3ff\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" Nov 04 05:27:47.420442 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.420427 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-trusted-ca-bundle\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:47.420545 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.420458 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-node-log\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.420634 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.420616 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-installation-pull-secrets\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:47.420741 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.420628 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4dcc3e21-060d-4d88-a034-025ee61af3ff-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-pbhxj\" (UID: \"4dcc3e21-060d-4d88-a034-025ee61af3ff\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" Nov 04 05:27:47.420814 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.420755 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-log-socket\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.420814 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.420784 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e1fc5a00-d482-4aa6-955d-ce6ff425240e-env-overrides\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.420909 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.420816 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9909a4b9-77f0-4be2-9374-26a602a72987-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:47.420909 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.420899 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-cnibin\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:27:47.420985 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.420925 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:27:47.420985 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.420942 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-service-ca\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:47.420985 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.420959 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bb1115f-702d-4b53-8852-4b38e452edb6-tmp-dir\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:47.420985 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.420982 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-sysctl-conf\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.421184 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.420998 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e1fc5a00-d482-4aa6-955d-ce6ff425240e-env-overrides\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.421184 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421008 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9909a4b9-77f0-4be2-9374-26a602a72987-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:47.421184 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421051 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.421184 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421079 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9909a4b9-77f0-4be2-9374-26a602a72987-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:47.421184 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421090 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf3a5689-535b-464f-9668-8d389d6396c5-tmp-dir\") pod \"node-resolver-7b7zs\" (UID: \"bf3a5689-535b-464f-9668-8d389d6396c5\") " pod="openshift-dns/node-resolver-7b7zs" Nov 04 05:27:47.421184 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421161 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1136554-e895-435d-a306-02744ddefdb2-cert\") pod \"ingress-canary-g2hdc\" (UID: \"a1136554-e895-435d-a306-02744ddefdb2\") " pod="openshift-ingress-canary/ingress-canary-g2hdc" Nov 04 05:27:47.421416 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421219 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-run-k8s-cni-cncf-io\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.421416 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421263 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.421416 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421280 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:27:47.421416 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421289 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-metrics-client-ca\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.421416 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421342 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf3a5689-535b-464f-9668-8d389d6396c5-tmp-dir\") pod \"node-resolver-7b7zs\" (UID: \"bf3a5689-535b-464f-9668-8d389d6396c5\") " pod="openshift-dns/node-resolver-7b7zs" Nov 04 05:27:47.421416 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421371 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-wtmp\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:47.421416 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421379 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1136554-e895-435d-a306-02744ddefdb2-cert\") pod \"ingress-canary-g2hdc\" (UID: \"a1136554-e895-435d-a306-02744ddefdb2\") " pod="openshift-ingress-canary/ingress-canary-g2hdc" Nov 04 05:27:47.421416 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421392 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-hostroot\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.421683 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421509 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmnb5\" (UniqueName: \"kubernetes.io/projected/5bb1115f-702d-4b53-8852-4b38e452edb6-kube-api-access-rmnb5\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:47.421683 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421537 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:27:47.421683 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421564 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2bc5\" (UniqueName: \"kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5\") pod \"network-check-target-cmc7w\" (UID: \"23cf3ade-4fac-44f1-813c-7f61427e2b83\") " pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:27:47.421683 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421590 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c4e843c-e392-42ed-b842-508a67b0f64b-host-slash\") pod \"iptables-alerter-mbh6p\" (UID: \"6c4e843c-e392-42ed-b842-508a67b0f64b\") " pod="openshift-network-operator/iptables-alerter-mbh6p" Nov 04 05:27:47.421683 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421615 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11b27b07-7ada-441c-8799-e3d3d111bc18-console-oauth-config\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:47.421683 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421653 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-cni-netd\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.421683 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421676 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-lib-modules\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.421913 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421699 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11b27b07-7ada-441c-8799-e3d3d111bc18-tmp\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:47.421913 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421723 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-sys-fs\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:27:47.421913 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421747 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-kubelet\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.421913 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421789 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsqms\" (UniqueName: \"kubernetes.io/projected/e1fc5a00-d482-4aa6-955d-ce6ff425240e-kube-api-access-rsqms\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.421913 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421816 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzngz\" (UniqueName: \"kubernetes.io/projected/4fc979bb-e663-4465-a2cc-16cb06be4963-kube-api-access-nzngz\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.421913 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421842 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9909a4b9-77f0-4be2-9374-26a602a72987-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:47.421913 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421873 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-config-volume\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.421913 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421895 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d3ea1a41-8e39-4d16-ae87-0e030773dc09-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.422192 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421924 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7810845b-042a-4fe6-bc58-cf7285f7e396-metrics-client-ca\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:47.422192 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421948 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s58tj\" (UniqueName: \"kubernetes.io/projected/11b27b07-7ada-441c-8799-e3d3d111bc18-kube-api-access-s58tj\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:47.422192 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421972 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-var-lib-kubelet\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.422192 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.421997 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:47.422192 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422049 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-bound-sa-token\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:47.422192 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422073 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7810845b-042a-4fe6-bc58-cf7285f7e396-sys\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:47.422192 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422096 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-var-lib-cni-multus\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.422192 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422120 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-run-multus-certs\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.422192 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422149 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-etc-selinux\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:27:47.422192 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422172 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-cni-bin\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.422494 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422195 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-run\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.422494 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422226 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d7741ae3-0846-4449-a738-45251ad287df-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-pnm9q\" (UID: \"d7741ae3-0846-4449-a738-45251ad287df\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pnm9q" Nov 04 05:27:47.422494 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422260 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-ca-trust-extracted\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:47.422494 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422285 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/83db651d-42a9-48ba-a7ef-263e32d237d9-agent-certs\") pod \"konnectivity-agent-z2pjx\" (UID: \"83db651d-42a9-48ba-a7ef-263e32d237d9\") " pod="kube-system/konnectivity-agent-z2pjx" Nov 04 05:27:47.422494 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422312 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-accelerators-collector-config\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:47.422494 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422338 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9909a4b9-77f0-4be2-9374-26a602a72987-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:47.422494 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422363 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-registry-tls\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:47.422494 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422387 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-tmp\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.422494 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422412 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-etc-kubernetes\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.422494 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422437 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-registration-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:27:47.422494 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422491 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-etc-openvswitch\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.422830 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422519 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvp55\" (UniqueName: \"kubernetes.io/projected/82e24a0f-8277-484a-b7b4-50dcfb2b4fdd-kube-api-access-zvp55\") pod \"downloads-6dd789c8b7-f59mk\" (UID: \"82e24a0f-8277-484a-b7b4-50dcfb2b4fdd\") " pod="openshift-console/downloads-6dd789c8b7-f59mk" Nov 04 05:27:47.422830 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422547 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq2fx\" (UniqueName: \"kubernetes.io/projected/9909a4b9-77f0-4be2-9374-26a602a72987-kube-api-access-cq2fx\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:47.422830 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422575 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.422830 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422601 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xftgr\" (UniqueName: \"kubernetes.io/projected/6c4e843c-e392-42ed-b842-508a67b0f64b-kube-api-access-xftgr\") pod \"iptables-alerter-mbh6p\" (UID: \"6c4e843c-e392-42ed-b842-508a67b0f64b\") " pod="openshift-network-operator/iptables-alerter-mbh6p" Nov 04 05:27:47.422830 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422626 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-cni-dir\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.422830 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422652 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs\") pod \"network-metrics-daemon-chl9q\" (UID: \"b0e60cf0-1764-4c18-ab4e-da874feaff36\") " pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:27:47.422830 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422681 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tcm2\" (UniqueName: \"kubernetes.io/projected/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-kube-api-access-2tcm2\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:47.422830 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422707 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-tls\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:47.422830 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422732 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.422830 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422759 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82e24a0f-8277-484a-b7b4-50dcfb2b4fdd-tmp\") pod \"downloads-6dd789c8b7-f59mk\" (UID: \"82e24a0f-8277-484a-b7b4-50dcfb2b4fdd\") " pod="openshift-console/downloads-6dd789c8b7-f59mk" Nov 04 05:27:47.422830 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422787 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-registry-certificates\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:47.422830 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422814 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jpd5\" (UniqueName: \"kubernetes.io/projected/7810845b-042a-4fe6-bc58-cf7285f7e396-kube-api-access-5jpd5\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:47.423193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422839 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-cnibin\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.423193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422865 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-device-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:27:47.423193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422889 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-run-ovn\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.423193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422914 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9baf3dfd-f665-42ed-8cad-5f4571bf7194-crio-socket\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:47.423193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422939 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-run-netns\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.423193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422964 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1fc5a00-d482-4aa6-955d-ce6ff425240e-ovn-node-metrics-cert\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.423193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.422995 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-sysctl-d\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.423193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423018 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-host\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.423193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423062 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9df1fb81-e74f-45cb-b07a-026f323567b5-serviceca\") pod \"node-ca-9bgz9\" (UID: \"9df1fb81-e74f-45cb-b07a-026f323567b5\") " pod="openshift-image-registry/node-ca-9bgz9" Nov 04 05:27:47.423193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423087 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-cni-binary-copy\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:27:47.423193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423116 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.423193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423143 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-image-registry-private-configuration\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:47.423193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423175 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx7s6\" (UniqueName: \"kubernetes.io/projected/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-kube-api-access-nx7s6\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.423621 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423201 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-socket-dir-parent\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.423621 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423226 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:27:47.423621 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423254 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4dcc3e21-060d-4d88-a034-025ee61af3ff-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-pbhxj\" (UID: \"4dcc3e21-060d-4d88-a034-025ee61af3ff\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" Nov 04 05:27:47.423621 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423288 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-run-systemd\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.423621 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423315 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e1fc5a00-d482-4aa6-955d-ce6ff425240e-ovnkube-config\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.423621 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423340 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1a41-8e39-4d16-ae87-0e030773dc09-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.423621 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423370 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqkfd\" (UniqueName: \"kubernetes.io/projected/a1136554-e895-435d-a306-02744ddefdb2-kube-api-access-vqkfd\") pod \"ingress-canary-g2hdc\" (UID: \"a1136554-e895-435d-a306-02744ddefdb2\") " pod="openshift-ingress-canary/ingress-canary-g2hdc" Nov 04 05:27:47.423621 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423398 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11b27b07-7ada-441c-8799-e3d3d111bc18-console-serving-cert\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:47.423621 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423423 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ac2e1d23-6505-4936-9c05-70a76f8fd886-cni-binary-copy\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.423621 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423450 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-daemon-config\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.423621 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423478 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b7cc\" (UniqueName: \"kubernetes.io/projected/00c77006-a012-4f0e-a886-69ff0150de6d-kube-api-access-6b7cc\") pod \"migrator-66df5d496b-hgld4\" (UID: \"00c77006-a012-4f0e-a886-69ff0150de6d\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-hgld4" Nov 04 05:27:47.423621 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423505 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-system-cni-dir\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:27:47.423621 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423530 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7810845b-042a-4fe6-bc58-cf7285f7e396-root\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:47.423621 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423557 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1a41-8e39-4d16-ae87-0e030773dc09-config-out\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.423621 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423583 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3ea1a41-8e39-4d16-ae87-0e030773dc09-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.423621 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423608 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5a710da8-8925-4e16-b43d-fc06a999569b-dbus\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:27:47.424095 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423634 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-run-netns\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.424095 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423669 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.424095 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423697 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.424095 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423723 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5a710da8-8925-4e16-b43d-fc06a999569b-kubelet-config\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:27:47.424095 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423750 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-console-config\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:47.424095 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423776 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/4a120e88-95d0-4176-ab05-586d4c2e7dab-archives-path\") pod \"periodic-gathering-tfj9k-j9jq9\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:27:47.424095 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423802 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-systemd-units\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.424095 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423828 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-run-openvswitch\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.424095 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423862 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9baf3dfd-f665-42ed-8cad-5f4571bf7194-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:47.424095 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423890 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.424095 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423921 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lhhb\" (UniqueName: \"kubernetes.io/projected/bf3a5689-535b-464f-9668-8d389d6396c5-kube-api-access-4lhhb\") pod \"node-resolver-7b7zs\" (UID: \"bf3a5689-535b-464f-9668-8d389d6396c5\") " pod="openshift-dns/node-resolver-7b7zs" Nov 04 05:27:47.424095 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423948 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-var-lib-cni-bin\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.424095 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.423976 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:27:47.424095 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.424006 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5bb1115f-702d-4b53-8852-4b38e452edb6-metrics-tls\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:47.424095 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.424047 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-trusted-ca\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:47.424095 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.424075 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fc979bb-e663-4465-a2cc-16cb06be4963-tmp\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.424095 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.424086 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bb1115f-702d-4b53-8852-4b38e452edb6-tmp-dir\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:47.424603 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.424101 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/83db651d-42a9-48ba-a7ef-263e32d237d9-konnectivity-ca\") pod \"konnectivity-agent-z2pjx\" (UID: \"83db651d-42a9-48ba-a7ef-263e32d237d9\") " pod="kube-system/konnectivity-agent-z2pjx" Nov 04 05:27:47.424603 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.424130 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5tw5\" (UniqueName: \"kubernetes.io/projected/ac2e1d23-6505-4936-9c05-70a76f8fd886-kube-api-access-f5tw5\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.424603 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.424157 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-var-lib-openvswitch\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.424603 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.424184 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-tuned\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.424603 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.424211 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6c4e843c-e392-42ed-b842-508a67b0f64b-iptables-alerter-script\") pod \"iptables-alerter-mbh6p\" (UID: \"6c4e843c-e392-42ed-b842-508a67b0f64b\") " pod="openshift-network-operator/iptables-alerter-mbh6p" Nov 04 05:27:47.424603 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.424239 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-system-cni-dir\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.424603 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.424265 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-sysconfig\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.424603 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.424279 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-registry-certificates\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:47.424603 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.424292 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-var-lib-kubelet\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.424900 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.424716 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1fc5a00-d482-4aa6-955d-ce6ff425240e-ovn-node-metrics-cert\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.425017 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.424999 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9909a4b9-77f0-4be2-9374-26a602a72987-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:47.425199 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.425184 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11b27b07-7ada-441c-8799-e3d3d111bc18-console-serving-cert\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:47.425245 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.425221 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ac2e1d23-6505-4936-9c05-70a76f8fd886-cni-binary-copy\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.425414 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.425399 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-daemon-config\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.425691 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.425675 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d7741ae3-0846-4449-a738-45251ad287df-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-pnm9q\" (UID: \"d7741ae3-0846-4449-a738-45251ad287df\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pnm9q" Nov 04 05:27:47.425784 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.425757 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/4a120e88-95d0-4176-ab05-586d4c2e7dab-archives-path\") pod \"periodic-gathering-tfj9k-j9jq9\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:27:47.425878 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.425828 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-ca-trust-extracted\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:47.425935 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.425883 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/83db651d-42a9-48ba-a7ef-263e32d237d9-agent-certs\") pod \"konnectivity-agent-z2pjx\" (UID: \"83db651d-42a9-48ba-a7ef-263e32d237d9\") " pod="kube-system/konnectivity-agent-z2pjx" Nov 04 05:27:47.425995 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.425976 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9909a4b9-77f0-4be2-9374-26a602a72987-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:47.426083 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426065 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9baf3dfd-f665-42ed-8cad-5f4571bf7194-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:47.426142 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426111 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a120e88-95d0-4176-ab05-586d4c2e7dab-trusted-ca-bundle\") pod \"periodic-gathering-tfj9k-j9jq9\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:27:47.426217 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426150 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-web-config\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.426217 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426175 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4fc979bb-e663-4465-a2cc-16cb06be4963-tmp\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.426371 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426235 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-tuned\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.426371 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426288 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/83db651d-42a9-48ba-a7ef-263e32d237d9-konnectivity-ca\") pod \"konnectivity-agent-z2pjx\" (UID: \"83db651d-42a9-48ba-a7ef-263e32d237d9\") " pod="kube-system/konnectivity-agent-z2pjx" Nov 04 05:27:47.426371 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426284 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.426371 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426348 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.426556 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426368 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-registry-tls\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:47.426556 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426382 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bf3a5689-535b-464f-9668-8d389d6396c5-hosts-file\") pod \"node-resolver-7b7zs\" (UID: \"bf3a5689-535b-464f-9668-8d389d6396c5\") " pod="openshift-dns/node-resolver-7b7zs" Nov 04 05:27:47.426556 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426368 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11b27b07-7ada-441c-8799-e3d3d111bc18-tmp\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:47.426556 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426422 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-textfile\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:47.426556 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426453 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5bb1115f-702d-4b53-8852-4b38e452edb6-metrics-tls\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:47.426556 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426454 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-run-ovn-kubernetes\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.426556 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426490 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-modprobe-d\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.426556 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426528 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-oauth-serving-cert\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:47.426556 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426546 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0e60cf0-1764-4c18-ab4e-da874feaff36-metrics-certs\") pod \"network-metrics-daemon-chl9q\" (UID: \"b0e60cf0-1764-4c18-ab4e-da874feaff36\") " pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:27:47.426978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426560 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-secret-grpc-tls\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.426978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426584 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:27:47.426978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426590 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:47.426978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426602 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-textfile\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:47.426978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426620 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-sys\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.426978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426640 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4dcc3e21-060d-4d88-a034-025ee61af3ff-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-pbhxj\" (UID: \"4dcc3e21-060d-4d88-a034-025ee61af3ff\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" Nov 04 05:27:47.426978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426649 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9df1fb81-e74f-45cb-b07a-026f323567b5-host\") pod \"node-ca-9bgz9\" (UID: \"9df1fb81-e74f-45cb-b07a-026f323567b5\") " pod="openshift-image-registry/node-ca-9bgz9" Nov 04 05:27:47.426978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426685 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hwxg\" (UniqueName: \"kubernetes.io/projected/9df1fb81-e74f-45cb-b07a-026f323567b5-kube-api-access-8hwxg\") pod \"node-ca-9bgz9\" (UID: \"9df1fb81-e74f-45cb-b07a-026f323567b5\") " pod="openshift-image-registry/node-ca-9bgz9" Nov 04 05:27:47.426978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426714 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h89x7\" (UniqueName: \"kubernetes.io/projected/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-api-access-h89x7\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:47.426978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426722 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-image-registry-private-configuration\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:47.426978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426746 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3ea1a41-8e39-4d16-ae87-0e030773dc09-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.426978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426774 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-secret-thanos-querier-tls\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.426978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426810 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9909a4b9-77f0-4be2-9374-26a602a72987-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:47.426978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426845 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vklnx\" (UniqueName: \"kubernetes.io/projected/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-kube-api-access-vklnx\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:27:47.426978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426863 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-tls\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:47.426978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426877 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9baf3dfd-f665-42ed-8cad-5f4571bf7194-data-volume\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:47.426978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426904 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82e24a0f-8277-484a-b7b4-50dcfb2b4fdd-tmp\") pod \"downloads-6dd789c8b7-f59mk\" (UID: \"82e24a0f-8277-484a-b7b4-50dcfb2b4fdd\") " pod="openshift-console/downloads-6dd789c8b7-f59mk" Nov 04 05:27:47.426978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426969 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-cni-binary-copy\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:27:47.426978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426974 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9df1fb81-e74f-45cb-b07a-026f323567b5-serviceca\") pod \"node-ca-9bgz9\" (UID: \"9df1fb81-e74f-45cb-b07a-026f323567b5\") " pod="openshift-image-registry/node-ca-9bgz9" Nov 04 05:27:47.426978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.426971 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:27:47.427811 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.427025 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-socket-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:27:47.427811 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.427080 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpqzl\" (UniqueName: \"kubernetes.io/projected/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-kube-api-access-mpqzl\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:27:47.427811 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.427111 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5bb1115f-702d-4b53-8852-4b38e452edb6-config-volume\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:47.427811 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.427201 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11b27b07-7ada-441c-8799-e3d3d111bc18-console-oauth-config\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:47.427811 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.427213 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e1fc5a00-d482-4aa6-955d-ce6ff425240e-ovnkube-config\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.427811 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.427272 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:47.427811 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.427287 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9baf3dfd-f665-42ed-8cad-5f4571bf7194-data-volume\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:47.427811 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.427311 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a710da8-8925-4e16-b43d-fc06a999569b-original-pull-secret\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:27:47.427811 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.427404 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bll88\" (UniqueName: \"kubernetes.io/projected/4a120e88-95d0-4176-ab05-586d4c2e7dab-kube-api-access-bll88\") pod \"periodic-gathering-tfj9k-j9jq9\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:27:47.427811 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.427452 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6c4e843c-e392-42ed-b842-508a67b0f64b-iptables-alerter-script\") pod \"iptables-alerter-mbh6p\" (UID: \"6c4e843c-e392-42ed-b842-508a67b0f64b\") " pod="openshift-network-operator/iptables-alerter-mbh6p" Nov 04 05:27:47.430379 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.430364 5258 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Nov 04 05:27:47.463407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.463332 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Nov 04 05:27:47.463407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.463365 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Nov 04 05:27:47.463579 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.463490 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Nov 04 05:27:47.464098 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.464074 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Nov 04 05:27:47.464098 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.464083 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Nov 04 05:27:47.464259 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.464120 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Nov 04 05:27:47.464259 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.464138 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Nov 04 05:27:47.464259 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.464229 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Nov 04 05:27:47.464386 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.464258 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Nov 04 05:27:47.464386 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.464267 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Nov 04 05:27:47.464697 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.464665 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Nov 04 05:27:47.464697 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.464676 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Nov 04 05:27:47.464697 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.464677 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Nov 04 05:27:47.464859 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.464839 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Nov 04 05:27:47.464902 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.464865 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Nov 04 05:27:47.464932 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.464914 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Nov 04 05:27:47.465179 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.465167 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Nov 04 05:27:47.465381 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.465364 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Nov 04 05:27:47.465810 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.465579 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:47.465927 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.465897 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-console-config\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:47.466172 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.466147 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-accelerators-collector-config\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:47.467206 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.467189 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-oauth-serving-cert\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:47.467394 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.467377 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9909a4b9-77f0-4be2-9374-26a602a72987-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:47.467692 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.467677 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5bb1115f-702d-4b53-8852-4b38e452edb6-config-volume\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:47.470746 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.470728 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a120e88-95d0-4176-ab05-586d4c2e7dab-service-ca-bundle\") pod \"periodic-gathering-tfj9k-j9jq9\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:27:47.470940 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.470924 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4dcc3e21-060d-4d88-a034-025ee61af3ff-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-pbhxj\" (UID: \"4dcc3e21-060d-4d88-a034-025ee61af3ff\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" Nov 04 05:27:47.471548 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.471528 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9909a4b9-77f0-4be2-9374-26a602a72987-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:47.471636 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.471559 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-service-ca\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:47.475790 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.475770 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7810845b-042a-4fe6-bc58-cf7285f7e396-metrics-client-ca\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:47.511403 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.511371 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Nov 04 05:27:47.511403 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.511371 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Nov 04 05:27:47.511594 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.511403 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Nov 04 05:27:47.511825 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.511806 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Nov 04 05:27:47.512228 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.512210 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nptl\" (UniqueName: \"kubernetes.io/projected/b0e60cf0-1764-4c18-ab4e-da874feaff36-kube-api-access-4nptl\") pod \"network-metrics-daemon-chl9q\" (UID: \"b0e60cf0-1764-4c18-ab4e-da874feaff36\") " pod="openshift-multus/network-metrics-daemon-chl9q" Nov 04 05:27:47.512524 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.512505 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzngz\" (UniqueName: \"kubernetes.io/projected/4fc979bb-e663-4465-a2cc-16cb06be4963-kube-api-access-nzngz\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.512591 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.512571 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvqg7\" (UniqueName: \"kubernetes.io/projected/4dcc3e21-060d-4d88-a034-025ee61af3ff-kube-api-access-tvqg7\") pod \"openshift-state-metrics-7bd7489957-pbhxj\" (UID: \"4dcc3e21-060d-4d88-a034-025ee61af3ff\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" Nov 04 05:27:47.512667 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.512651 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jpd5\" (UniqueName: \"kubernetes.io/projected/7810845b-042a-4fe6-bc58-cf7285f7e396-kube-api-access-5jpd5\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:47.512931 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.512910 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2bc5\" (UniqueName: \"kubernetes.io/projected/23cf3ade-4fac-44f1-813c-7f61427e2b83-kube-api-access-r2bc5\") pod \"network-check-target-cmc7w\" (UID: \"23cf3ade-4fac-44f1-813c-7f61427e2b83\") " pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:27:47.513081 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.513066 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hwxg\" (UniqueName: \"kubernetes.io/projected/9df1fb81-e74f-45cb-b07a-026f323567b5-kube-api-access-8hwxg\") pod \"node-ca-9bgz9\" (UID: \"9df1fb81-e74f-45cb-b07a-026f323567b5\") " pod="openshift-image-registry/node-ca-9bgz9" Nov 04 05:27:47.513120 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.513094 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b7cc\" (UniqueName: \"kubernetes.io/projected/00c77006-a012-4f0e-a886-69ff0150de6d-kube-api-access-6b7cc\") pod \"migrator-66df5d496b-hgld4\" (UID: \"00c77006-a012-4f0e-a886-69ff0150de6d\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-hgld4" Nov 04 05:27:47.513363 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.513342 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bll88\" (UniqueName: \"kubernetes.io/projected/4a120e88-95d0-4176-ab05-586d4c2e7dab-kube-api-access-bll88\") pod \"periodic-gathering-tfj9k-j9jq9\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:27:47.513428 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.513407 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqkfd\" (UniqueName: \"kubernetes.io/projected/a1136554-e895-435d-a306-02744ddefdb2-kube-api-access-vqkfd\") pod \"ingress-canary-g2hdc\" (UID: \"a1136554-e895-435d-a306-02744ddefdb2\") " pod="openshift-ingress-canary/ingress-canary-g2hdc" Nov 04 05:27:47.513460 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.513422 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lhhb\" (UniqueName: \"kubernetes.io/projected/bf3a5689-535b-464f-9668-8d389d6396c5-kube-api-access-4lhhb\") pod \"node-resolver-7b7zs\" (UID: \"bf3a5689-535b-464f-9668-8d389d6396c5\") " pod="openshift-dns/node-resolver-7b7zs" Nov 04 05:27:47.513506 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.513491 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5tw5\" (UniqueName: \"kubernetes.io/projected/ac2e1d23-6505-4936-9c05-70a76f8fd886-kube-api-access-f5tw5\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.513552 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.513533 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-bound-sa-token\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:47.513918 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.513903 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h89x7\" (UniqueName: \"kubernetes.io/projected/9baf3dfd-f665-42ed-8cad-5f4571bf7194-kube-api-access-h89x7\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:47.513949 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.513915 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsqms\" (UniqueName: \"kubernetes.io/projected/e1fc5a00-d482-4aa6-955d-ce6ff425240e-kube-api-access-rsqms\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.513980 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.513942 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xftgr\" (UniqueName: \"kubernetes.io/projected/6c4e843c-e392-42ed-b842-508a67b0f64b-kube-api-access-xftgr\") pod \"iptables-alerter-mbh6p\" (UID: \"6c4e843c-e392-42ed-b842-508a67b0f64b\") " pod="openshift-network-operator/iptables-alerter-mbh6p" Nov 04 05:27:47.513980 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.513946 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmnb5\" (UniqueName: \"kubernetes.io/projected/5bb1115f-702d-4b53-8852-4b38e452edb6-kube-api-access-rmnb5\") pod \"dns-default-8fgr7\" (UID: \"5bb1115f-702d-4b53-8852-4b38e452edb6\") " pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:47.514119 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.514098 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpqzl\" (UniqueName: \"kubernetes.io/projected/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-kube-api-access-mpqzl\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:27:47.514561 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.514541 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vklnx\" (UniqueName: \"kubernetes.io/projected/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-kube-api-access-vklnx\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:27:47.514631 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.514616 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s58tj\" (UniqueName: \"kubernetes.io/projected/11b27b07-7ada-441c-8799-e3d3d111bc18-kube-api-access-s58tj\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:47.514768 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.514754 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq2fx\" (UniqueName: \"kubernetes.io/projected/9909a4b9-77f0-4be2-9374-26a602a72987-kube-api-access-cq2fx\") pod \"kube-state-metrics-5dc8d7d89b-p2pjj\" (UID: \"9909a4b9-77f0-4be2-9374-26a602a72987\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" Nov 04 05:27:47.514805 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.514757 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvp55\" (UniqueName: \"kubernetes.io/projected/82e24a0f-8277-484a-b7b4-50dcfb2b4fdd-kube-api-access-zvp55\") pod \"downloads-6dd789c8b7-f59mk\" (UID: \"82e24a0f-8277-484a-b7b4-50dcfb2b4fdd\") " pod="openshift-console/downloads-6dd789c8b7-f59mk" Nov 04 05:27:47.514805 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.514787 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tcm2\" (UniqueName: \"kubernetes.io/projected/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-kube-api-access-2tcm2\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:47.520883 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.517625 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a120e88-95d0-4176-ab05-586d4c2e7dab-trusted-ca-bundle\") pod \"periodic-gathering-tfj9k-j9jq9\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:27:47.520883 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.517643 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4806f73f-d8d3-41a3-865f-9df7fe88e1b8-trusted-ca\") pod \"image-registry-74f7d9b7f7-488q9\" (UID: \"4806f73f-d8d3-41a3-865f-9df7fe88e1b8\") " pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:47.521545 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.521524 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-trusted-ca-bundle\") pod \"console-788f8bd759-f6zjh\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:27:47.528292 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528270 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-systemd\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.528388 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528298 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-kubernetes\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.528388 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528320 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-node-log\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.528388 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528342 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-log-socket\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.528388 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528361 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-cnibin\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:27:47.528388 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528378 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-sysctl-conf\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.528573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528394 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.528573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528407 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-kubernetes\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.528573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528414 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-run-k8s-cni-cncf-io\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.528573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528437 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-node-log\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.528573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528445 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-run-k8s-cni-cncf-io\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.528573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528393 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-systemd\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.528573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528462 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.528573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528483 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-log-socket\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.528573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528489 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-metrics-client-ca\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.528573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528509 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-cnibin\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:27:47.528573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528514 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-wtmp\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:47.528573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528535 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-hostroot\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.528573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528560 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c4e843c-e392-42ed-b842-508a67b0f64b-host-slash\") pod \"iptables-alerter-mbh6p\" (UID: \"6c4e843c-e392-42ed-b842-508a67b0f64b\") " pod="openshift-network-operator/iptables-alerter-mbh6p" Nov 04 05:27:47.528573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528581 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-cni-netd\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.529108 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528599 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-lib-modules\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.529108 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528622 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-sys-fs\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:27:47.529108 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528637 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-sysctl-conf\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.529108 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528644 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-kubelet\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.529108 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528671 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-config-volume\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.529108 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528693 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d3ea1a41-8e39-4d16-ae87-0e030773dc09-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.529108 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528716 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-var-lib-kubelet\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.529108 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528741 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7810845b-042a-4fe6-bc58-cf7285f7e396-sys\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:47.529108 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528765 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-var-lib-cni-multus\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.529108 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528791 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-run-multus-certs\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.529108 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528796 5258 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Nov 04 05:27:47.529108 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528806 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-kubelet\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.529108 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528813 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-etc-selinux\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:27:47.529108 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528868 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-etc-selinux\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:27:47.529108 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528871 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-cni-bin\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.529108 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528902 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-run\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.529108 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528906 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-cni-bin\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.529108 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528947 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c4e843c-e392-42ed-b842-508a67b0f64b-host-slash\") pod \"iptables-alerter-mbh6p\" (UID: \"6c4e843c-e392-42ed-b842-508a67b0f64b\") " pod="openshift-network-operator/iptables-alerter-mbh6p" Nov 04 05:27:47.529108 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.528671 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-hostroot\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.529864 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.529346 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-metrics-client-ca\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.529864 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.529491 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-cni-netd\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.529864 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.529608 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-lib-modules\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.529864 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.529657 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-sys-fs\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:27:47.529864 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.529705 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7810845b-042a-4fe6-bc58-cf7285f7e396-node-exporter-wtmp\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:47.529864 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.529740 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7810845b-042a-4fe6-bc58-cf7285f7e396-sys\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:47.530159 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530125 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-var-lib-kubelet\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.530205 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530170 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-run-multus-certs\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.530236 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530203 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-var-lib-cni-multus\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.530287 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530251 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-run\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.530333 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530297 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-tmp\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.530384 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530350 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-etc-kubernetes\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.530384 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530373 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-registration-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:27:47.530483 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530408 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-etc-openvswitch\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.530483 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530436 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.530559 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530479 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-cni-dir\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.530559 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530510 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.530559 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530553 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-cnibin\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.530695 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530578 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-device-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:27:47.530695 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530601 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-run-ovn\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.530695 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530626 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9baf3dfd-f665-42ed-8cad-5f4571bf7194-crio-socket\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:47.530695 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530652 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-run-netns\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.530855 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530705 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-sysctl-d\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.530855 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530728 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-host\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.530855 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530756 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.530855 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530782 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx7s6\" (UniqueName: \"kubernetes.io/projected/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-kube-api-access-nx7s6\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.530855 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530807 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-socket-dir-parent\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.530855 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530829 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:27:47.530855 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530853 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-run-systemd\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.531164 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530894 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-run-netns\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.531164 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.530925 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.531327 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.531302 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9baf3dfd-f665-42ed-8cad-5f4571bf7194-crio-socket\") pod \"insights-runtime-extractor-qrxks\" (UID: \"9baf3dfd-f665-42ed-8cad-5f4571bf7194\") " pod="openshift-insights/insights-runtime-extractor-qrxks" Nov 04 05:27:47.531386 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.531361 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-registration-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:27:47.531438 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.531405 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-device-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:27:47.531497 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.531446 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-cnibin\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.531497 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.531479 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-run-ovn\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.531592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.531525 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-cni-dir\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.531592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.531561 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-etc-kubernetes\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.531712 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.531680 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-etc-openvswitch\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.531712 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.531693 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-tmp\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.531861 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.531757 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-host\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.531861 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.531805 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.531956 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.531875 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-sysctl-d\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.531956 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.531886 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:27:47.531956 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.531911 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1a41-8e39-4d16-ae87-0e030773dc09-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.531956 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.531927 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-socket-dir-parent\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.532173 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.531981 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-system-cni-dir\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:27:47.532173 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.531998 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-run-systemd\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.532173 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532005 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7810845b-042a-4fe6-bc58-cf7285f7e396-root\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:47.532173 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532020 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1a41-8e39-4d16-ae87-0e030773dc09-config-out\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.532173 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532064 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3ea1a41-8e39-4d16-ae87-0e030773dc09-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.532173 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532095 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5a710da8-8925-4e16-b43d-fc06a999569b-dbus\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:27:47.532173 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532125 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-run-netns\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.532173 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532151 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.532537 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532177 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.532537 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532202 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5a710da8-8925-4e16-b43d-fc06a999569b-kubelet-config\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:27:47.532537 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532230 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-systemd-units\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.532537 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532257 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-run-openvswitch\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.532537 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532284 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.532537 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532309 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-var-lib-cni-bin\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.532537 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532333 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:27:47.532537 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532391 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-var-lib-openvswitch\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.532537 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532420 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-system-cni-dir\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.532537 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532446 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-sysconfig\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.532537 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532472 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-var-lib-kubelet\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.532537 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532494 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-web-config\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.532537 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532519 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.533143 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532547 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.533143 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532574 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bf3a5689-535b-464f-9668-8d389d6396c5-hosts-file\") pod \"node-resolver-7b7zs\" (UID: \"bf3a5689-535b-464f-9668-8d389d6396c5\") " pod="openshift-dns/node-resolver-7b7zs" Nov 04 05:27:47.533143 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532598 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-run-ovn-kubernetes\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.533143 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532622 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-modprobe-d\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.533143 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532645 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-secret-grpc-tls\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.533143 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532668 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-sys\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.533143 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532691 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9df1fb81-e74f-45cb-b07a-026f323567b5-host\") pod \"node-ca-9bgz9\" (UID: \"9df1fb81-e74f-45cb-b07a-026f323567b5\") " pod="openshift-image-registry/node-ca-9bgz9" Nov 04 05:27:47.533143 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532717 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3ea1a41-8e39-4d16-ae87-0e030773dc09-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.533143 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532736 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-secret-thanos-querier-tls\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.533143 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532755 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-socket-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:27:47.533143 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532774 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-slash\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.533143 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532790 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqt7m\" (UniqueName: \"kubernetes.io/projected/d3ea1a41-8e39-4d16-ae87-0e030773dc09-kube-api-access-bqt7m\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.533143 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532804 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-conf-dir\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.533143 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532877 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-multus-conf-dir\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.534125 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.533165 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.534125 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.533249 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5a710da8-8925-4e16-b43d-fc06a999569b-dbus\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:27:47.534125 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.533319 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-run-netns\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.534125 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.533447 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-config-volume\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.534125 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.533508 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-host-var-lib-cni-bin\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.534125 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.533559 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5a710da8-8925-4e16-b43d-fc06a999569b-kubelet-config\") pod \"global-pull-secret-syncer-4pvss\" (UID: \"5a710da8-8925-4e16-b43d-fc06a999569b\") " pod="kube-system/global-pull-secret-syncer-4pvss" Nov 04 05:27:47.534125 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.533599 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-systemd-units\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.534125 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.532230 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1a41-8e39-4d16-ae87-0e030773dc09-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.534125 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.533656 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-run-openvswitch\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.534125 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.533969 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.534125 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.534047 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-system-cni-dir\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:27:47.534125 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.534087 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7810845b-042a-4fe6-bc58-cf7285f7e396-root\") pod \"node-exporter-69gbq\" (UID: \"7810845b-042a-4fe6-bc58-cf7285f7e396\") " pod="openshift-monitoring/node-exporter-69gbq" Nov 04 05:27:47.534125 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.534134 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-sys\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.534762 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.534157 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1a41-8e39-4d16-ae87-0e030773dc09-config-out\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.534762 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.534199 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-run-ovn-kubernetes\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.534762 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.534208 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bf3a5689-535b-464f-9668-8d389d6396c5-hosts-file\") pod \"node-resolver-7b7zs\" (UID: \"bf3a5689-535b-464f-9668-8d389d6396c5\") " pod="openshift-dns/node-resolver-7b7zs" Nov 04 05:27:47.534762 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.534296 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-modprobe-d\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.534762 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.534710 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.535029 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.534791 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ac2e1d23-6505-4936-9c05-70a76f8fd886-system-cni-dir\") pod \"multus-nn7ng\" (UID: \"ac2e1d23-6505-4936-9c05-70a76f8fd886\") " pod="openshift-multus/multus-nn7ng" Nov 04 05:27:47.535029 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.534839 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3ea1a41-8e39-4d16-ae87-0e030773dc09-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.535029 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.534847 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99a606bd-b4a1-43e9-8525-a7afcd1eae0f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bzbgk\" (UID: \"99a606bd-b4a1-43e9-8525-a7afcd1eae0f\") " pod="openshift-multus/multus-additional-cni-plugins-bzbgk" Nov 04 05:27:47.535029 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.534885 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-var-lib-openvswitch\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.537132 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.535360 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e1fc5a00-d482-4aa6-955d-ce6ff425240e-host-slash\") pod \"ovnkube-node-cnn7v\" (UID: \"e1fc5a00-d482-4aa6-955d-ce6ff425240e\") " pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:47.537132 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.535381 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-var-lib-kubelet\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.537132 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.535432 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050-socket-dir\") pod \"aws-ebs-csi-driver-node-vhv6z\" (UID: \"7ca23bb0-8c76-4d96-8ab5-4b57dbcb1050\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vhv6z" Nov 04 05:27:47.537132 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.535449 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4fc979bb-e663-4465-a2cc-16cb06be4963-etc-sysconfig\") pod \"tuned-dff2s\" (UID: \"4fc979bb-e663-4465-a2cc-16cb06be4963\") " pod="openshift-cluster-node-tuning-operator/tuned-dff2s" Nov 04 05:27:47.537132 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.535711 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9df1fb81-e74f-45cb-b07a-026f323567b5-host\") pod \"node-ca-9bgz9\" (UID: \"9df1fb81-e74f-45cb-b07a-026f323567b5\") " pod="openshift-image-registry/node-ca-9bgz9" Nov 04 05:27:47.537132 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.536356 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.537132 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.536644 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3ea1a41-8e39-4d16-ae87-0e030773dc09-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.537132 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.536691 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d3ea1a41-8e39-4d16-ae87-0e030773dc09-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.537132 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.536964 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-secret-grpc-tls\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.537132 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.537091 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.537617 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.537188 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.537617 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.537519 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.537721 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.537617 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-secret-thanos-querier-tls\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.537889 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.537872 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-web-config\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.537956 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.537939 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.590787 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.590750 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx7s6\" (UniqueName: \"kubernetes.io/projected/4f8aadd2-cf8e-47a7-9931-04701ad5c34f-kube-api-access-nx7s6\") pod \"thanos-querier-75d996c778-7l2qj\" (UID: \"4f8aadd2-cf8e-47a7-9931-04701ad5c34f\") " pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:47.597159 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.597135 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqt7m\" (UniqueName: \"kubernetes.io/projected/d3ea1a41-8e39-4d16-ae87-0e030773dc09-kube-api-access-bqt7m\") pod \"alertmanager-main-0\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.626437 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.626412 5258 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 05:27:47.629239 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.629219 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:27:47.651610 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:47.651433 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:48.064923 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.064858 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 05:27:48.067336 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:48.067293 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3ea1a41_8e39_4d16_ae87_0e030773dc09.slice/crio-5db860688e5dd154c8977c208ee2f94578f3c26bba2b40fb148efa4cd6e99790 WatchSource:0}: Error finding container 5db860688e5dd154c8977c208ee2f94578f3c26bba2b40fb148efa4cd6e99790: Status 404 returned error can't find the container with id 5db860688e5dd154c8977c208ee2f94578f3c26bba2b40fb148efa4cd6e99790 Nov 04 05:27:48.101182 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.101093 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-75d996c778-7l2qj"] Nov 04 05:27:48.104146 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:48.104119 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f8aadd2_cf8e_47a7_9931_04701ad5c34f.slice/crio-e5ab9db1f1492cdbfb24553411e00a2ddf62efaa3e389867c6b2302e1d9c8122 WatchSource:0}: Error finding container e5ab9db1f1492cdbfb24553411e00a2ddf62efaa3e389867c6b2302e1d9c8122: Status 404 returned error can't find the container with id e5ab9db1f1492cdbfb24553411e00a2ddf62efaa3e389867c6b2302e1d9c8122 Nov 04 05:27:48.292648 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.292618 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-dcqtd"] Nov 04 05:27:48.295626 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.295607 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-dcqtd" Nov 04 05:27:48.338992 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.338860 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc7gj\" (UniqueName: \"kubernetes.io/projected/07d855d9-bce9-42be-9acf-df2386b300ab-kube-api-access-hc7gj\") pod \"monitoring-plugin-77f4f76594-dcqtd\" (UID: \"07d855d9-bce9-42be-9acf-df2386b300ab\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-dcqtd" Nov 04 05:27:48.338992 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.338983 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/07d855d9-bce9-42be-9acf-df2386b300ab-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-dcqtd\" (UID: \"07d855d9-bce9-42be-9acf-df2386b300ab\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-dcqtd" Nov 04 05:27:48.349864 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.349829 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" event={"ID":"4f8aadd2-cf8e-47a7-9931-04701ad5c34f","Type":"ContainerStarted","Data":"e5ab9db1f1492cdbfb24553411e00a2ddf62efaa3e389867c6b2302e1d9c8122"} Nov 04 05:27:48.350017 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.349939 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-dcqtd"] Nov 04 05:27:48.352567 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.352530 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3ea1a41-8e39-4d16-ae87-0e030773dc09","Type":"ContainerStarted","Data":"5db860688e5dd154c8977c208ee2f94578f3c26bba2b40fb148efa4cd6e99790"} Nov 04 05:27:48.403893 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.403861 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-2q75z\"" Nov 04 05:27:48.403893 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.403876 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Nov 04 05:27:48.439359 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.439328 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc7gj\" (UniqueName: \"kubernetes.io/projected/07d855d9-bce9-42be-9acf-df2386b300ab-kube-api-access-hc7gj\") pod \"monitoring-plugin-77f4f76594-dcqtd\" (UID: \"07d855d9-bce9-42be-9acf-df2386b300ab\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-dcqtd" Nov 04 05:27:48.439592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.439375 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/07d855d9-bce9-42be-9acf-df2386b300ab-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-dcqtd\" (UID: \"07d855d9-bce9-42be-9acf-df2386b300ab\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-dcqtd" Nov 04 05:27:48.442210 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.442187 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/07d855d9-bce9-42be-9acf-df2386b300ab-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-dcqtd\" (UID: \"07d855d9-bce9-42be-9acf-df2386b300ab\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-dcqtd" Nov 04 05:27:48.498261 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.498230 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc7gj\" (UniqueName: \"kubernetes.io/projected/07d855d9-bce9-42be-9acf-df2386b300ab-kube-api-access-hc7gj\") pod \"monitoring-plugin-77f4f76594-dcqtd\" (UID: \"07d855d9-bce9-42be-9acf-df2386b300ab\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-dcqtd" Nov 04 05:27:48.608427 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.608401 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-dcqtd" Nov 04 05:27:48.667774 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.667741 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-78bc77fdbf-nrvds"] Nov 04 05:27:48.671059 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.670993 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:48.721382 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.721355 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Nov 04 05:27:48.727188 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.727153 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-78bc77fdbf-nrvds"] Nov 04 05:27:48.788406 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.788374 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5zzj6\"" Nov 04 05:27:48.788582 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.788421 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-g5qvcihnqcff\"" Nov 04 05:27:48.788582 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.788472 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Nov 04 05:27:48.788582 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.788493 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Nov 04 05:27:48.821295 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.821266 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Nov 04 05:27:48.844767 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.844679 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxzdv\" (UniqueName: \"kubernetes.io/projected/5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2-kube-api-access-gxzdv\") pod \"metrics-server-78bc77fdbf-nrvds\" (UID: \"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2\") " pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:48.844767 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.844748 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78bc77fdbf-nrvds\" (UID: \"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2\") " pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:48.844998 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.844840 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2-metrics-server-audit-profiles\") pod \"metrics-server-78bc77fdbf-nrvds\" (UID: \"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2\") " pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:48.844998 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.844893 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2-audit-log\") pod \"metrics-server-78bc77fdbf-nrvds\" (UID: \"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2\") " pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:48.844998 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.844925 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2-secret-metrics-server-tls\") pod \"metrics-server-78bc77fdbf-nrvds\" (UID: \"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2\") " pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:48.844998 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.844969 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2-secret-metrics-server-client-certs\") pod \"metrics-server-78bc77fdbf-nrvds\" (UID: \"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2\") " pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:48.845205 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.845002 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2-client-ca-bundle\") pod \"metrics-server-78bc77fdbf-nrvds\" (UID: \"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2\") " pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:48.848417 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.848374 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8fgr7" Nov 04 05:27:48.946254 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.945477 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxzdv\" (UniqueName: \"kubernetes.io/projected/5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2-kube-api-access-gxzdv\") pod \"metrics-server-78bc77fdbf-nrvds\" (UID: \"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2\") " pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:48.946254 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.945545 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78bc77fdbf-nrvds\" (UID: \"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2\") " pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:48.946254 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.945583 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2-metrics-server-audit-profiles\") pod \"metrics-server-78bc77fdbf-nrvds\" (UID: \"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2\") " pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:48.946254 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.945631 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2-audit-log\") pod \"metrics-server-78bc77fdbf-nrvds\" (UID: \"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2\") " pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:48.946254 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.945661 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2-secret-metrics-server-tls\") pod \"metrics-server-78bc77fdbf-nrvds\" (UID: \"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2\") " pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:48.946254 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.945721 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2-secret-metrics-server-client-certs\") pod \"metrics-server-78bc77fdbf-nrvds\" (UID: \"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2\") " pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:48.946254 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.945749 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2-client-ca-bundle\") pod \"metrics-server-78bc77fdbf-nrvds\" (UID: \"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2\") " pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:48.946739 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.946480 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2-audit-log\") pod \"metrics-server-78bc77fdbf-nrvds\" (UID: \"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2\") " pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:48.947155 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.947131 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-78bc77fdbf-nrvds\" (UID: \"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2\") " pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:48.947749 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.947722 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2-metrics-server-audit-profiles\") pod \"metrics-server-78bc77fdbf-nrvds\" (UID: \"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2\") " pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:48.950723 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.950696 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2-client-ca-bundle\") pod \"metrics-server-78bc77fdbf-nrvds\" (UID: \"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2\") " pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:48.951327 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.951294 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2-secret-metrics-server-client-certs\") pod \"metrics-server-78bc77fdbf-nrvds\" (UID: \"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2\") " pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:48.952257 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.952236 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2-secret-metrics-server-tls\") pod \"metrics-server-78bc77fdbf-nrvds\" (UID: \"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2\") " pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:48.991306 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:48.991269 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-dcqtd"] Nov 04 05:27:48.996964 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:48.996928 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07d855d9_bce9_42be_9acf_df2386b300ab.slice/crio-f904ae942f40227dd276c9dfae6e8b41a6282b8513b9a625b969650b46500a0e WatchSource:0}: Error finding container f904ae942f40227dd276c9dfae6e8b41a6282b8513b9a625b969650b46500a0e: Status 404 returned error can't find the container with id f904ae942f40227dd276c9dfae6e8b41a6282b8513b9a625b969650b46500a0e Nov 04 05:27:49.011703 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:49.011675 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxzdv\" (UniqueName: \"kubernetes.io/projected/5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2-kube-api-access-gxzdv\") pod \"metrics-server-78bc77fdbf-nrvds\" (UID: \"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2\") " pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:49.284821 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:49.284783 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:27:49.365694 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:49.365604 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-dcqtd" event={"ID":"07d855d9-bce9-42be-9acf-df2386b300ab","Type":"ContainerStarted","Data":"f904ae942f40227dd276c9dfae6e8b41a6282b8513b9a625b969650b46500a0e"} Nov 04 05:27:49.370176 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:49.368995 5258 generic.go:356] "Generic (PLEG): container finished" podID="7810845b-042a-4fe6-bc58-cf7285f7e396" containerID="4b7cf4b6d190ea5d66274129192e4331731e9d7922e402ef230c22a1ae442d84" exitCode=0 Nov 04 05:27:49.370176 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:49.369065 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-69gbq" event={"ID":"7810845b-042a-4fe6-bc58-cf7285f7e396","Type":"ContainerDied","Data":"4b7cf4b6d190ea5d66274129192e4331731e9d7922e402ef230c22a1ae442d84"} Nov 04 05:27:49.669623 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:49.669427 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-78bc77fdbf-nrvds"] Nov 04 05:27:50.398553 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:50.398512 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-69gbq" event={"ID":"7810845b-042a-4fe6-bc58-cf7285f7e396","Type":"ContainerStarted","Data":"120e3c5f0ed1ad75f212ce5288222f2ce7408ee93dc1dd04363268c2dfb13e3d"} Nov 04 05:27:50.399126 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:50.398614 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-69gbq" event={"ID":"7810845b-042a-4fe6-bc58-cf7285f7e396","Type":"ContainerStarted","Data":"bdda55a8e29f0d08c9c6ae3ef98851d6d9a4125528bfae969ceaeca4bed9a28c"} Nov 04 05:27:50.400983 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:50.400956 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" event={"ID":"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2","Type":"ContainerStarted","Data":"b99b3533b3f02640ff01b82b76b12f963e3b4f60a22e069103d448a111659949"} Nov 04 05:27:50.604640 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:50.604566 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:51.409106 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:51.408952 5258 generic.go:356] "Generic (PLEG): container finished" podID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerID="1d221231c4d9e99a74ebc078aab66afda3ab84ebe52355ceec8651bde41c3f98" exitCode=0 Nov 04 05:27:51.409106 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:51.409014 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3ea1a41-8e39-4d16-ae87-0e030773dc09","Type":"ContainerDied","Data":"1d221231c4d9e99a74ebc078aab66afda3ab84ebe52355ceec8651bde41c3f98"} Nov 04 05:27:51.426467 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:51.425743 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pbhxj" event={"ID":"4dcc3e21-060d-4d88-a034-025ee61af3ff","Type":"ContainerStarted","Data":"f02014d1a549f4dafe2990e580d66840e75cbc24e9f0d273780822d6f00726db"} Nov 04 05:27:51.430748 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:51.430721 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" event={"ID":"9909a4b9-77f0-4be2-9374-26a602a72987","Type":"ContainerStarted","Data":"d02905fbd5440cfc56346ef6d5ecfb473cc6d1cb042a91c3d8c4df8f205f833f"} Nov 04 05:27:51.430850 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:51.430776 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" event={"ID":"9909a4b9-77f0-4be2-9374-26a602a72987","Type":"ContainerStarted","Data":"a67d9ef23899b2efe9276265d2fec0efc7909b6576233bb26ca16a43b6aed9d7"} Nov 04 05:27:51.430850 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:51.430807 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p2pjj" event={"ID":"9909a4b9-77f0-4be2-9374-26a602a72987","Type":"ContainerStarted","Data":"e48a72feb3e1ad15cbb26da252b4271647ae481af5bbcfbfe1abb7133325daa3"} Nov 04 05:27:52.014011 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.013303 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-74f7d9b7f7-488q9" Nov 04 05:27:52.437771 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.437697 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-dcqtd" event={"ID":"07d855d9-bce9-42be-9acf-df2386b300ab","Type":"ContainerStarted","Data":"3cb2a6aea176ede1e844b08e3ea01b3ad40c3539f82b3c03135eb2e869b1bf66"} Nov 04 05:27:52.440490 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.440466 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-dcqtd" Nov 04 05:27:52.446622 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.446162 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-dcqtd" Nov 04 05:27:52.668458 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.668405 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-dcqtd" podStartSLOduration=1.400383979 podStartE2EDuration="4.668390152s" totalImagesPullingTime="3.268006173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:27:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:52.55596946 +0000 UTC m=+6.435361934" watchObservedRunningTime="2026-11-04 05:27:52.668390152 +0000 UTC m=+6.547782625" Nov 04 05:27:52.787087 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:27:52.786948 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:27:52.854351 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.854310 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 05:27:52.879252 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.878643 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.894722 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.894688 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.894860 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.894736 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.894860 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.894837 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.894946 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.894880 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-config\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.894946 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.894901 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84b76\" (UniqueName: \"kubernetes.io/projected/e42acc16-8c50-4990-8f7e-35ac3df287f2-kube-api-access-84b76\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.895071 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.895042 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.895200 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.895109 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.895200 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.895149 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.895309 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.895241 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.895309 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.895289 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e42acc16-8c50-4990-8f7e-35ac3df287f2-config-out\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.895411 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.895314 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.895411 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.895338 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.895411 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.895372 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.895562 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.895398 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.895562 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.895457 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-web-config\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.895562 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.895480 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.895562 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.895513 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.895562 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.895536 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.895790 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.895571 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e42acc16-8c50-4990-8f7e-35ac3df287f2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.895790 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.895599 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.922588 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.922523 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 05:27:52.932466 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.932429 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Nov 04 05:27:52.932615 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.932440 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Nov 04 05:27:52.932615 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.932564 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Nov 04 05:27:52.932615 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.932586 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Nov 04 05:27:52.932728 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.932655 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-mjb6f\"" Nov 04 05:27:52.932728 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.932715 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-e13puemqhc024\"" Nov 04 05:27:52.932789 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.932752 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Nov 04 05:27:52.932847 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.932831 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Nov 04 05:27:52.932954 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.932718 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Nov 04 05:27:52.933014 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.932926 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Nov 04 05:27:52.933014 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.932834 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Nov 04 05:27:52.933173 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.933009 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Nov 04 05:27:52.933173 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.933123 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Nov 04 05:27:52.933364 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.933292 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Nov 04 05:27:52.980749 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.980510 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Nov 04 05:27:52.980749 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.980710 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Nov 04 05:27:52.996527 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.996494 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.996663 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.996541 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.996663 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.996571 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.996663 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.996608 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-config\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.996663 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.996634 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84b76\" (UniqueName: \"kubernetes.io/projected/e42acc16-8c50-4990-8f7e-35ac3df287f2-kube-api-access-84b76\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.996663 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.996658 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.996917 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.996681 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.996917 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.996707 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.996917 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.996746 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.996917 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.996789 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e42acc16-8c50-4990-8f7e-35ac3df287f2-config-out\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.996917 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.996816 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.996917 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.996841 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.996917 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.996884 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.996917 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.996912 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.997339 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.996936 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-web-config\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.997339 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.996963 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.997339 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.996997 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.997339 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.997024 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.997339 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.997092 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e42acc16-8c50-4990-8f7e-35ac3df287f2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.997339 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.997122 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.998505 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.998438 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.998839 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.998807 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:52.998920 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.998858 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:53.000629 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:52.999759 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:53.001635 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.001605 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:53.002421 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.002395 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:53.002904 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.002758 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:53.003408 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.003375 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:53.004224 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.004176 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:53.004625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.004600 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:53.005222 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.005205 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-pnm9q" Nov 04 05:27:53.007074 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.006480 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-config\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:53.007169 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.007147 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e42acc16-8c50-4990-8f7e-35ac3df287f2-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:53.008082 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.008059 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:53.008663 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.008632 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:53.009195 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.009152 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:53.009712 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.009650 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:53.010225 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.010173 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-web-config\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:53.010570 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.010531 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e42acc16-8c50-4990-8f7e-35ac3df287f2-config-out\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:53.013849 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.013828 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:53.062937 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.062865 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84b76\" (UniqueName: \"kubernetes.io/projected/e42acc16-8c50-4990-8f7e-35ac3df287f2-kube-api-access-84b76\") pod \"prometheus-k8s-0\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:53.078355 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.078167 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:27:53.455067 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.454388 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" event={"ID":"4f8aadd2-cf8e-47a7-9931-04701ad5c34f","Type":"ContainerStarted","Data":"b4f4f98cda63c591872fde90c4fbe435ad7145b44bdafd42f10ec7779601fad0"} Nov 04 05:27:53.455067 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.454523 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" event={"ID":"4f8aadd2-cf8e-47a7-9931-04701ad5c34f","Type":"ContainerStarted","Data":"2090e65683d907f1a811989d67d4f3513e1485d38f9ba718c4e64b088b1a0931"} Nov 04 05:27:53.455067 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.454579 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" event={"ID":"4f8aadd2-cf8e-47a7-9931-04701ad5c34f","Type":"ContainerStarted","Data":"67331e70e177e08ca2e30285c28e76cc98f66e74052b7792c15f74674aea7b7f"} Nov 04 05:27:53.459143 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.459076 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" event={"ID":"5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2","Type":"ContainerStarted","Data":"a77210d204d1108ae247d5157c4fd745e02c5a15f34e53a311d02ecb9cc7dae9"} Nov 04 05:27:53.549151 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.549119 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 05:27:53.552499 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:27:53.552459 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode42acc16_8c50_4990_8f7e_35ac3df287f2.slice/crio-30117051ab1899940396e42201ab0a4de429c3ae79bbe5164e134aaf76dd3660 WatchSource:0}: Error finding container 30117051ab1899940396e42201ab0a4de429c3ae79bbe5164e134aaf76dd3660: Status 404 returned error can't find the container with id 30117051ab1899940396e42201ab0a4de429c3ae79bbe5164e134aaf76dd3660 Nov 04 05:27:53.802939 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.802914 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:27:53.803336 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:53.803246 5258 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 05:27:54.000931 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:54.000897 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:27:54.116537 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:54.116480 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" podStartSLOduration=2.8321426560000003 podStartE2EDuration="6.116463534s" totalImagesPullingTime="3.284320878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:27:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:53.573163943 +0000 UTC m=+7.452556417" watchObservedRunningTime="2026-11-04 05:27:54.116463534 +0000 UTC m=+7.995856006" Nov 04 05:27:54.388442 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:27:54.388388 5258 log.go:32] "PullImage from image service failed" err="rpc error: code = Canceled desc = unable to pull image: copying system image from manifest list: copying config: context canceled" image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:36c3da5821b9e51334e9c523aca7fca6a9ef8b853fd4f238b908b086b129619d" Nov 04 05:27:54.388715 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:27:54.388692 5258 kuberuntime_manager.go:1707] "Unhandled Error" err="container download-server start failed in pod downloads-6dd789c8b7-f59mk_openshift-console(82e24a0f-8277-484a-b7b4-50dcfb2b4fdd): ErrImagePull: rpc error: code = Canceled desc = unable to pull image: copying system image from manifest list: copying config: context canceled" logger="UnhandledError" Nov 04 05:27:54.388801 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:27:54.388736 5258 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"download-server\" with ErrImagePull: \"rpc error: code = Canceled desc = unable to pull image: copying system image from manifest list: copying config: context canceled\"" pod="openshift-console/downloads-6dd789c8b7-f59mk" podUID="82e24a0f-8277-484a-b7b4-50dcfb2b4fdd" Nov 04 05:27:54.464726 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:54.464693 5258 generic.go:356] "Generic (PLEG): container finished" podID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerID="38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31" exitCode=0 Nov 04 05:27:54.465210 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:54.464783 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e42acc16-8c50-4990-8f7e-35ac3df287f2","Type":"ContainerDied","Data":"38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31"} Nov 04 05:27:54.465210 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:54.464870 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e42acc16-8c50-4990-8f7e-35ac3df287f2","Type":"ContainerStarted","Data":"30117051ab1899940396e42201ab0a4de429c3ae79bbe5164e134aaf76dd3660"} Nov 04 05:27:54.466673 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:54.466644 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-788f8bd759-f6zjh" event={"ID":"11b27b07-7ada-441c-8799-e3d3d111bc18","Type":"ContainerStarted","Data":"d89e0a5ff8acd41e94c688e13fe0d53283effad98da082f3fa27d77af3e80777"} Nov 04 05:27:55.473701 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:55.473662 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3ea1a41-8e39-4d16-ae87-0e030773dc09","Type":"ContainerStarted","Data":"5645f1cbfc23dbf7f1df2d6f1b25d59fff57826b88df18717240801d7f55b48a"} Nov 04 05:27:55.474286 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:55.473775 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3ea1a41-8e39-4d16-ae87-0e030773dc09","Type":"ContainerStarted","Data":"3303030c71badc37f6cac49fb374fc7a0522a4b1557d3f3def975b0b5155a0f6"} Nov 04 05:27:55.474286 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:55.473819 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3ea1a41-8e39-4d16-ae87-0e030773dc09","Type":"ContainerStarted","Data":"8ae33980838366191d5dbd5fa627365bd0b36e72cd72aecb95467135682de5eb"} Nov 04 05:27:55.474286 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:55.473863 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3ea1a41-8e39-4d16-ae87-0e030773dc09","Type":"ContainerStarted","Data":"75bd935f2e20d910c08be90e3209141921bdf4f0b90d06593296a5af6295f0b0"} Nov 04 05:27:55.474286 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:55.473900 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3ea1a41-8e39-4d16-ae87-0e030773dc09","Type":"ContainerStarted","Data":"64848ccfcbea01b7a469debaf158fc72758eadd1a01036c1ebb288daea728e6d"} Nov 04 05:27:55.476620 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:55.476595 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" event={"ID":"4f8aadd2-cf8e-47a7-9931-04701ad5c34f","Type":"ContainerStarted","Data":"18a32d0ffa914f2ee730e62cf77a44478bcdaf145d9c32e3c29c90a5d89cdd4c"} Nov 04 05:27:55.476716 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:55.476651 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" event={"ID":"4f8aadd2-cf8e-47a7-9931-04701ad5c34f","Type":"ContainerStarted","Data":"0fc079fc8950d581623bf91b670ca1c4f398debfabb14c76501e9acf4d49313b"} Nov 04 05:27:55.476716 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:55.476685 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" event={"ID":"4f8aadd2-cf8e-47a7-9931-04701ad5c34f","Type":"ContainerStarted","Data":"f94702fb42ba30ca2063b71345166dba26fc6ec15884492de0bce81888cb43f6"} Nov 04 05:27:55.476790 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:55.476765 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:27:55.595748 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:55.595702 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" podStartSLOduration=3.0700585 podStartE2EDuration="9.595688567s" totalImagesPullingTime="6.525630067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:27:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:55.59477062 +0000 UTC m=+9.474163092" watchObservedRunningTime="2026-11-04 05:27:55.595688567 +0000 UTC m=+9.475081077" Nov 04 05:27:56.484390 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:56.484308 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3ea1a41-8e39-4d16-ae87-0e030773dc09","Type":"ContainerStarted","Data":"91e78c938b656502c5de651aa3a03e936abb1ff70d36673caef95f20f37afe7b"} Nov 04 05:27:56.608497 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:56.608443 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=5.14190441 podStartE2EDuration="11.608428155s" totalImagesPullingTime="6.447759698s" totalInitContainerRuntime="18.764047ms" isStatefulPod=false podCreationTimestamp="2026-11-04 05:27:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:56.607679696 +0000 UTC m=+10.487072180" watchObservedRunningTime="2026-11-04 05:27:56.608428155 +0000 UTC m=+10.487820643" Nov 04 05:27:57.581765 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:57.581728 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-z2pjx" Nov 04 05:27:58.977388 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:58.977361 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-788f8bd759-f6zjh"] Nov 04 05:27:59.501489 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:59.501403 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e42acc16-8c50-4990-8f7e-35ac3df287f2","Type":"ContainerStarted","Data":"8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d"} Nov 04 05:27:59.501489 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:59.501477 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e42acc16-8c50-4990-8f7e-35ac3df287f2","Type":"ContainerStarted","Data":"5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7"} Nov 04 05:27:59.501692 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:59.501516 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e42acc16-8c50-4990-8f7e-35ac3df287f2","Type":"ContainerStarted","Data":"f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6"} Nov 04 05:27:59.501692 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:59.501550 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e42acc16-8c50-4990-8f7e-35ac3df287f2","Type":"ContainerStarted","Data":"96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554"} Nov 04 05:27:59.501692 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:59.501583 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e42acc16-8c50-4990-8f7e-35ac3df287f2","Type":"ContainerStarted","Data":"63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294"} Nov 04 05:27:59.501692 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:59.501616 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e42acc16-8c50-4990-8f7e-35ac3df287f2","Type":"ContainerStarted","Data":"85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526"} Nov 04 05:27:59.632748 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:27:59.629229 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.493827038 podStartE2EDuration="7.62920948s" totalImagesPullingTime="4.114916879s" totalInitContainerRuntime="20.465563ms" isStatefulPod=false podCreationTimestamp="2026-11-04 05:27:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:27:59.627397548 +0000 UTC m=+13.506790034" watchObservedRunningTime="2026-11-04 05:27:59.62920948 +0000 UTC m=+13.508601955" Nov 04 05:28:01.490441 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:01.490411 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-75d996c778-7l2qj" Nov 04 05:28:03.079177 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:03.079139 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:04.100531 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:04.100490 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:28:04.707132 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:04.707096 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:28:10.546994 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:10.546955 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-f59mk" event={"ID":"82e24a0f-8277-484a-b7b4-50dcfb2b4fdd","Type":"ContainerStarted","Data":"cff3dd8a28a5909e2035daaa4c43363307c9c5e41e53affce86b500a66a6667f"} Nov 04 05:28:10.547511 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:10.547392 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-f59mk" Nov 04 05:28:10.561380 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:10.561344 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-f59mk" Nov 04 05:28:13.216659 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:13.216630 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:28:13.222866 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:13.222841 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:28:13.564283 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:13.564203 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-78bc77fdbf-nrvds" Nov 04 05:28:14.004898 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:14.004860 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-cmc7w" Nov 04 05:28:16.807607 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:16.807568 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cnn7v" Nov 04 05:28:22.783178 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:28:22.783137 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:28:23.997443 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:23.997386 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-788f8bd759-f6zjh" podUID="11b27b07-7ada-441c-8799-e3d3d111bc18" containerName="console" containerID="cri-o://d89e0a5ff8acd41e94c688e13fe0d53283effad98da082f3fa27d77af3e80777" gracePeriod=15 Nov 04 05:28:24.294930 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.294900 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:28:24.399098 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.399060 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/11b27b07-7ada-441c-8799-e3d3d111bc18-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11b27b07-7ada-441c-8799-e3d3d111bc18-console-oauth-config\") pod \"11b27b07-7ada-441c-8799-e3d3d111bc18\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " Nov 04 05:28:24.399098 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.399106 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11b27b07-7ada-441c-8799-e3d3d111bc18-tmp\" (UniqueName: \"kubernetes.io/empty-dir/11b27b07-7ada-441c-8799-e3d3d111bc18-tmp\") pod \"11b27b07-7ada-441c-8799-e3d3d111bc18\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " Nov 04 05:28:24.399307 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.399155 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11b27b07-7ada-441c-8799-e3d3d111bc18-kube-api-access-s58tj\" (UniqueName: \"kubernetes.io/projected/11b27b07-7ada-441c-8799-e3d3d111bc18-kube-api-access-s58tj\") pod \"11b27b07-7ada-441c-8799-e3d3d111bc18\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " Nov 04 05:28:24.399307 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.399196 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-service-ca\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-service-ca\") pod \"11b27b07-7ada-441c-8799-e3d3d111bc18\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " Nov 04 05:28:24.399307 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.399227 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/11b27b07-7ada-441c-8799-e3d3d111bc18-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11b27b07-7ada-441c-8799-e3d3d111bc18-console-serving-cert\") pod \"11b27b07-7ada-441c-8799-e3d3d111bc18\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " Nov 04 05:28:24.399307 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.399249 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-trusted-ca-bundle\") pod \"11b27b07-7ada-441c-8799-e3d3d111bc18\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " Nov 04 05:28:24.399307 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.399298 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-console-config\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-console-config\") pod \"11b27b07-7ada-441c-8799-e3d3d111bc18\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " Nov 04 05:28:24.399568 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.399337 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-oauth-serving-cert\") pod \"11b27b07-7ada-441c-8799-e3d3d111bc18\" (UID: \"11b27b07-7ada-441c-8799-e3d3d111bc18\") " Nov 04 05:28:24.399568 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.399440 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11b27b07-7ada-441c-8799-e3d3d111bc18-tmp" pod "11b27b07-7ada-441c-8799-e3d3d111bc18" (UID: "11b27b07-7ada-441c-8799-e3d3d111bc18"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:28:24.399683 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.399654 5258 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11b27b07-7ada-441c-8799-e3d3d111bc18-tmp\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:24.399958 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.399801 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-service-ca" pod "11b27b07-7ada-441c-8799-e3d3d111bc18" (UID: "11b27b07-7ada-441c-8799-e3d3d111bc18"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:24.399958 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.399894 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-trusted-ca-bundle" pod "11b27b07-7ada-441c-8799-e3d3d111bc18" (UID: "11b27b07-7ada-441c-8799-e3d3d111bc18"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:24.399958 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.399911 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-oauth-serving-cert" pod "11b27b07-7ada-441c-8799-e3d3d111bc18" (UID: "11b27b07-7ada-441c-8799-e3d3d111bc18"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:24.399958 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.399926 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-console-config" pod "11b27b07-7ada-441c-8799-e3d3d111bc18" (UID: "11b27b07-7ada-441c-8799-e3d3d111bc18"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:24.401604 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.401579 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11b27b07-7ada-441c-8799-e3d3d111bc18-console-oauth-config" pod "11b27b07-7ada-441c-8799-e3d3d111bc18" (UID: "11b27b07-7ada-441c-8799-e3d3d111bc18"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:24.401811 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.401784 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11b27b07-7ada-441c-8799-e3d3d111bc18-console-serving-cert" pod "11b27b07-7ada-441c-8799-e3d3d111bc18" (UID: "11b27b07-7ada-441c-8799-e3d3d111bc18"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:24.401858 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.401795 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11b27b07-7ada-441c-8799-e3d3d111bc18-kube-api-access-s58tj" pod "11b27b07-7ada-441c-8799-e3d3d111bc18" (UID: "11b27b07-7ada-441c-8799-e3d3d111bc18"). InnerVolumeSpecName "kube-api-access-s58tj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:28:24.500556 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.500522 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s58tj\" (UniqueName: \"kubernetes.io/projected/11b27b07-7ada-441c-8799-e3d3d111bc18-kube-api-access-s58tj\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:24.500556 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.500554 5258 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-service-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:24.500757 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.500566 5258 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11b27b07-7ada-441c-8799-e3d3d111bc18-console-serving-cert\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:24.500757 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.500575 5258 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-trusted-ca-bundle\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:24.500757 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.500584 5258 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-console-config\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:24.500757 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.500593 5258 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11b27b07-7ada-441c-8799-e3d3d111bc18-oauth-serving-cert\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:24.500757 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.500601 5258 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11b27b07-7ada-441c-8799-e3d3d111bc18-console-oauth-config\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:24.608372 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.608337 5258 generic.go:356] "Generic (PLEG): container finished" podID="11b27b07-7ada-441c-8799-e3d3d111bc18" containerID="d89e0a5ff8acd41e94c688e13fe0d53283effad98da082f3fa27d77af3e80777" exitCode=0 Nov 04 05:28:24.608550 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.608384 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-788f8bd759-f6zjh" Nov 04 05:28:24.608550 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.608418 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-788f8bd759-f6zjh" event={"ID":"11b27b07-7ada-441c-8799-e3d3d111bc18","Type":"ContainerDied","Data":"d89e0a5ff8acd41e94c688e13fe0d53283effad98da082f3fa27d77af3e80777"} Nov 04 05:28:24.608550 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.608467 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-788f8bd759-f6zjh" event={"ID":"11b27b07-7ada-441c-8799-e3d3d111bc18","Type":"ContainerDied","Data":"7af0d5e931332006d648b413e3d1c89f456a6b0dae1ab2e542ebad9416d483e4"} Nov 04 05:28:24.608550 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.608494 5258 scope.go:122] "RemoveContainer" containerID="d89e0a5ff8acd41e94c688e13fe0d53283effad98da082f3fa27d77af3e80777" Nov 04 05:28:24.617580 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.617563 5258 scope.go:122] "RemoveContainer" containerID="d89e0a5ff8acd41e94c688e13fe0d53283effad98da082f3fa27d77af3e80777" Nov 04 05:28:24.617831 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:28:24.617813 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d89e0a5ff8acd41e94c688e13fe0d53283effad98da082f3fa27d77af3e80777\": container with ID starting with d89e0a5ff8acd41e94c688e13fe0d53283effad98da082f3fa27d77af3e80777 not found: ID does not exist" containerID="d89e0a5ff8acd41e94c688e13fe0d53283effad98da082f3fa27d77af3e80777" Nov 04 05:28:24.617880 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.617840 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d89e0a5ff8acd41e94c688e13fe0d53283effad98da082f3fa27d77af3e80777"} err="failed to get container status \"d89e0a5ff8acd41e94c688e13fe0d53283effad98da082f3fa27d77af3e80777\": rpc error: code = NotFound desc = could not find container \"d89e0a5ff8acd41e94c688e13fe0d53283effad98da082f3fa27d77af3e80777\": container with ID starting with d89e0a5ff8acd41e94c688e13fe0d53283effad98da082f3fa27d77af3e80777 not found: ID does not exist" Nov 04 05:28:24.763725 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.763690 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-788f8bd759-f6zjh"] Nov 04 05:28:24.767426 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:24.767398 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-788f8bd759-f6zjh"] Nov 04 05:28:26.270608 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:26.270569 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="11b27b07-7ada-441c-8799-e3d3d111bc18" path="/var/lib/kubelet/pods/11b27b07-7ada-441c-8799-e3d3d111bc18/volumes" Nov 04 05:28:27.453508 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:27.453464 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-tfj9k-j9jq9"] Nov 04 05:28:27.453988 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:27.453717 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" podUID="4a120e88-95d0-4176-ab05-586d4c2e7dab" containerName="insights-gathering" containerID="cri-o://74533311ad9c9e7227db1a0d702ce84f353bbeb9c512d72ee3f827413142321b" gracePeriod=30 Nov 04 05:28:27.623215 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:27.623179 5258 generic.go:356] "Generic (PLEG): container finished" podID="4a120e88-95d0-4176-ab05-586d4c2e7dab" containerID="74533311ad9c9e7227db1a0d702ce84f353bbeb9c512d72ee3f827413142321b" exitCode=2 Nov 04 05:28:27.623341 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:27.623235 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" event={"ID":"4a120e88-95d0-4176-ab05-586d4c2e7dab","Type":"ContainerDied","Data":"74533311ad9c9e7227db1a0d702ce84f353bbeb9c512d72ee3f827413142321b"} Nov 04 05:28:27.710180 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:27.710126 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:28:27.829157 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:27.829123 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4a120e88-95d0-4176-ab05-586d4c2e7dab-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a120e88-95d0-4176-ab05-586d4c2e7dab-service-ca-bundle\") pod \"4a120e88-95d0-4176-ab05-586d4c2e7dab\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " Nov 04 05:28:27.829315 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:27.829191 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4a120e88-95d0-4176-ab05-586d4c2e7dab-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a120e88-95d0-4176-ab05-586d4c2e7dab-trusted-ca-bundle\") pod \"4a120e88-95d0-4176-ab05-586d4c2e7dab\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " Nov 04 05:28:27.829315 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:27.829290 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a120e88-95d0-4176-ab05-586d4c2e7dab-kube-api-access-bll88\" (UniqueName: \"kubernetes.io/projected/4a120e88-95d0-4176-ab05-586d4c2e7dab-kube-api-access-bll88\") pod \"4a120e88-95d0-4176-ab05-586d4c2e7dab\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " Nov 04 05:28:27.829416 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:27.829317 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a120e88-95d0-4176-ab05-586d4c2e7dab-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/4a120e88-95d0-4176-ab05-586d4c2e7dab-archives-path\") pod \"4a120e88-95d0-4176-ab05-586d4c2e7dab\" (UID: \"4a120e88-95d0-4176-ab05-586d4c2e7dab\") " Nov 04 05:28:27.829576 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:27.829544 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a120e88-95d0-4176-ab05-586d4c2e7dab-service-ca-bundle" pod "4a120e88-95d0-4176-ab05-586d4c2e7dab" (UID: "4a120e88-95d0-4176-ab05-586d4c2e7dab"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:27.829669 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:27.829561 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a120e88-95d0-4176-ab05-586d4c2e7dab-trusted-ca-bundle" pod "4a120e88-95d0-4176-ab05-586d4c2e7dab" (UID: "4a120e88-95d0-4176-ab05-586d4c2e7dab"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:27.829799 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:27.829779 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a120e88-95d0-4176-ab05-586d4c2e7dab-archives-path" pod "4a120e88-95d0-4176-ab05-586d4c2e7dab" (UID: "4a120e88-95d0-4176-ab05-586d4c2e7dab"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:28:27.831652 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:27.831630 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a120e88-95d0-4176-ab05-586d4c2e7dab-kube-api-access-bll88" pod "4a120e88-95d0-4176-ab05-586d4c2e7dab" (UID: "4a120e88-95d0-4176-ab05-586d4c2e7dab"). InnerVolumeSpecName "kube-api-access-bll88". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:28:27.930813 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:27.930778 5258 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a120e88-95d0-4176-ab05-586d4c2e7dab-trusted-ca-bundle\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:27.930813 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:27.930807 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bll88\" (UniqueName: \"kubernetes.io/projected/4a120e88-95d0-4176-ab05-586d4c2e7dab-kube-api-access-bll88\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:27.930813 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:27.930818 5258 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/4a120e88-95d0-4176-ab05-586d4c2e7dab-archives-path\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:27.931077 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:27.930827 5258 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a120e88-95d0-4176-ab05-586d4c2e7dab-service-ca-bundle\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:28.628751 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:28.628716 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" event={"ID":"4a120e88-95d0-4176-ab05-586d4c2e7dab","Type":"ContainerDied","Data":"fbaa1a138a1355400c58c2fbd44b7cd69611c765efe58c7e2ae239c8086b05c6"} Nov 04 05:28:28.629299 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:28.628783 5258 scope.go:122] "RemoveContainer" containerID="74533311ad9c9e7227db1a0d702ce84f353bbeb9c512d72ee3f827413142321b" Nov 04 05:28:28.629299 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:28.628729 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-tfj9k-j9jq9" Nov 04 05:28:28.783434 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:28.783399 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-tfj9k-j9jq9"] Nov 04 05:28:28.830692 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:28.830653 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-tfj9k-j9jq9"] Nov 04 05:28:30.267728 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:30.267695 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4a120e88-95d0-4176-ab05-586d4c2e7dab" path="/var/lib/kubelet/pods/4a120e88-95d0-4176-ab05-586d4c2e7dab/volumes" Nov 04 05:28:36.234923 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:36.234858 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:36.284889 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:36.284863 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:36.678764 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:36.678737 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:50.207202 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:50.207169 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 05:28:50.207711 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:50.207446 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="alertmanager" containerID="cri-o://64848ccfcbea01b7a469debaf158fc72758eadd1a01036c1ebb288daea728e6d" gracePeriod=120 Nov 04 05:28:50.207711 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:50.207520 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="kube-rbac-proxy-metric" containerID="cri-o://5645f1cbfc23dbf7f1df2d6f1b25d59fff57826b88df18717240801d7f55b48a" gracePeriod=120 Nov 04 05:28:50.207711 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:50.207548 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="config-reloader" containerID="cri-o://75bd935f2e20d910c08be90e3209141921bdf4f0b90d06593296a5af6295f0b0" gracePeriod=120 Nov 04 05:28:50.207711 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:50.207582 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="kube-rbac-proxy" containerID="cri-o://3303030c71badc37f6cac49fb374fc7a0522a4b1557d3f3def975b0b5155a0f6" gracePeriod=120 Nov 04 05:28:50.207711 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:50.207609 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="prom-label-proxy" containerID="cri-o://91e78c938b656502c5de651aa3a03e936abb1ff70d36673caef95f20f37afe7b" gracePeriod=120 Nov 04 05:28:50.207984 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:50.207535 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="kube-rbac-proxy-web" containerID="cri-o://8ae33980838366191d5dbd5fa627365bd0b36e72cd72aecb95467135682de5eb" gracePeriod=120 Nov 04 05:28:50.720423 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:50.720393 5258 generic.go:356] "Generic (PLEG): container finished" podID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerID="91e78c938b656502c5de651aa3a03e936abb1ff70d36673caef95f20f37afe7b" exitCode=0 Nov 04 05:28:50.720423 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:50.720416 5258 generic.go:356] "Generic (PLEG): container finished" podID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerID="5645f1cbfc23dbf7f1df2d6f1b25d59fff57826b88df18717240801d7f55b48a" exitCode=0 Nov 04 05:28:50.720423 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:50.720422 5258 generic.go:356] "Generic (PLEG): container finished" podID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerID="3303030c71badc37f6cac49fb374fc7a0522a4b1557d3f3def975b0b5155a0f6" exitCode=0 Nov 04 05:28:50.720423 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:50.720428 5258 generic.go:356] "Generic (PLEG): container finished" podID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerID="75bd935f2e20d910c08be90e3209141921bdf4f0b90d06593296a5af6295f0b0" exitCode=0 Nov 04 05:28:50.720423 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:50.720433 5258 generic.go:356] "Generic (PLEG): container finished" podID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerID="64848ccfcbea01b7a469debaf158fc72758eadd1a01036c1ebb288daea728e6d" exitCode=0 Nov 04 05:28:50.720705 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:50.720461 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3ea1a41-8e39-4d16-ae87-0e030773dc09","Type":"ContainerDied","Data":"91e78c938b656502c5de651aa3a03e936abb1ff70d36673caef95f20f37afe7b"} Nov 04 05:28:50.720705 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:50.720524 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3ea1a41-8e39-4d16-ae87-0e030773dc09","Type":"ContainerDied","Data":"5645f1cbfc23dbf7f1df2d6f1b25d59fff57826b88df18717240801d7f55b48a"} Nov 04 05:28:50.720705 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:50.720551 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3ea1a41-8e39-4d16-ae87-0e030773dc09","Type":"ContainerDied","Data":"3303030c71badc37f6cac49fb374fc7a0522a4b1557d3f3def975b0b5155a0f6"} Nov 04 05:28:50.720705 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:50.720574 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3ea1a41-8e39-4d16-ae87-0e030773dc09","Type":"ContainerDied","Data":"75bd935f2e20d910c08be90e3209141921bdf4f0b90d06593296a5af6295f0b0"} Nov 04 05:28:50.720705 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:50.720600 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3ea1a41-8e39-4d16-ae87-0e030773dc09","Type":"ContainerDied","Data":"64848ccfcbea01b7a469debaf158fc72758eadd1a01036c1ebb288daea728e6d"} Nov 04 05:28:51.457754 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.457732 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.536201 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.536126 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d3ea1a41-8e39-4d16-ae87-0e030773dc09-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3ea1a41-8e39-4d16-ae87-0e030773dc09-metrics-client-ca\") pod \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " Nov 04 05:28:51.536201 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.536162 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-kube-rbac-proxy\") pod \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " Nov 04 05:28:51.536201 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.536185 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-cluster-tls-config\") pod \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " Nov 04 05:28:51.536432 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.536206 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-kube-rbac-proxy-web\") pod \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " Nov 04 05:28:51.536432 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.536229 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d3ea1a41-8e39-4d16-ae87-0e030773dc09-config-out\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1a41-8e39-4d16-ae87-0e030773dc09-config-out\") pod \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " Nov 04 05:28:51.536432 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.536358 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d3ea1a41-8e39-4d16-ae87-0e030773dc09-tls-assets\" (UniqueName: \"kubernetes.io/projected/d3ea1a41-8e39-4d16-ae87-0e030773dc09-tls-assets\") pod \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " Nov 04 05:28:51.536432 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.536413 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-main-tls\") pod \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " Nov 04 05:28:51.536644 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.536452 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-web-config\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-web-config\") pod \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " Nov 04 05:28:51.536644 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.536486 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-kube-rbac-proxy-metric\") pod \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " Nov 04 05:28:51.536644 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.536578 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d3ea1a41-8e39-4d16-ae87-0e030773dc09-kube-api-access-bqt7m\" (UniqueName: \"kubernetes.io/projected/d3ea1a41-8e39-4d16-ae87-0e030773dc09-kube-api-access-bqt7m\") pod \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " Nov 04 05:28:51.536644 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.536578 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3ea1a41-8e39-4d16-ae87-0e030773dc09-metrics-client-ca" pod "d3ea1a41-8e39-4d16-ae87-0e030773dc09" (UID: "d3ea1a41-8e39-4d16-ae87-0e030773dc09"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:51.536644 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.536626 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d3ea1a41-8e39-4d16-ae87-0e030773dc09-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1a41-8e39-4d16-ae87-0e030773dc09-alertmanager-main-db\") pod \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " Nov 04 05:28:51.536909 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.536658 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-config-volume\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-config-volume\") pod \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " Nov 04 05:28:51.536909 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.536705 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d3ea1a41-8e39-4d16-ae87-0e030773dc09-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3ea1a41-8e39-4d16-ae87-0e030773dc09-alertmanager-trusted-ca-bundle\") pod \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\" (UID: \"d3ea1a41-8e39-4d16-ae87-0e030773dc09\") " Nov 04 05:28:51.537061 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.536977 5258 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3ea1a41-8e39-4d16-ae87-0e030773dc09-metrics-client-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:51.537698 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.537366 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3ea1a41-8e39-4d16-ae87-0e030773dc09-alertmanager-main-db" pod "d3ea1a41-8e39-4d16-ae87-0e030773dc09" (UID: "d3ea1a41-8e39-4d16-ae87-0e030773dc09"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:28:51.537698 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.537404 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3ea1a41-8e39-4d16-ae87-0e030773dc09-alertmanager-trusted-ca-bundle" pod "d3ea1a41-8e39-4d16-ae87-0e030773dc09" (UID: "d3ea1a41-8e39-4d16-ae87-0e030773dc09"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:51.539129 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.539087 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3ea1a41-8e39-4d16-ae87-0e030773dc09-config-out" pod "d3ea1a41-8e39-4d16-ae87-0e030773dc09" (UID: "d3ea1a41-8e39-4d16-ae87-0e030773dc09"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:28:51.539492 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.539469 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-kube-rbac-proxy" pod "d3ea1a41-8e39-4d16-ae87-0e030773dc09" (UID: "d3ea1a41-8e39-4d16-ae87-0e030773dc09"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:51.539771 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.539748 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-kube-rbac-proxy-web" pod "d3ea1a41-8e39-4d16-ae87-0e030773dc09" (UID: "d3ea1a41-8e39-4d16-ae87-0e030773dc09"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:51.540306 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.540261 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3ea1a41-8e39-4d16-ae87-0e030773dc09-tls-assets" pod "d3ea1a41-8e39-4d16-ae87-0e030773dc09" (UID: "d3ea1a41-8e39-4d16-ae87-0e030773dc09"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:28:51.540418 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.540370 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-main-tls" pod "d3ea1a41-8e39-4d16-ae87-0e030773dc09" (UID: "d3ea1a41-8e39-4d16-ae87-0e030773dc09"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:51.540525 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.540498 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-config-volume" pod "d3ea1a41-8e39-4d16-ae87-0e030773dc09" (UID: "d3ea1a41-8e39-4d16-ae87-0e030773dc09"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:51.541118 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.541090 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-kube-rbac-proxy-metric" pod "d3ea1a41-8e39-4d16-ae87-0e030773dc09" (UID: "d3ea1a41-8e39-4d16-ae87-0e030773dc09"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:51.541193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.541129 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3ea1a41-8e39-4d16-ae87-0e030773dc09-kube-api-access-bqt7m" pod "d3ea1a41-8e39-4d16-ae87-0e030773dc09" (UID: "d3ea1a41-8e39-4d16-ae87-0e030773dc09"). InnerVolumeSpecName "kube-api-access-bqt7m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:28:51.543954 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.543929 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-cluster-tls-config" pod "d3ea1a41-8e39-4d16-ae87-0e030773dc09" (UID: "d3ea1a41-8e39-4d16-ae87-0e030773dc09"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:51.551186 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.551166 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-web-config" pod "d3ea1a41-8e39-4d16-ae87-0e030773dc09" (UID: "d3ea1a41-8e39-4d16-ae87-0e030773dc09"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:51.637822 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.637790 5258 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1a41-8e39-4d16-ae87-0e030773dc09-alertmanager-main-db\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:51.637822 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.637814 5258 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-config-volume\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:51.637822 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.637825 5258 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3ea1a41-8e39-4d16-ae87-0e030773dc09-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:51.638026 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.637835 5258 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:51.638026 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.637845 5258 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-cluster-tls-config\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:51.638026 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.637854 5258 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:51.638026 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.637862 5258 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d3ea1a41-8e39-4d16-ae87-0e030773dc09-config-out\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:51.638026 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.637870 5258 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d3ea1a41-8e39-4d16-ae87-0e030773dc09-tls-assets\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:51.638026 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.637878 5258 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-main-tls\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:51.638026 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.637888 5258 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-web-config\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:51.638026 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.637898 5258 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d3ea1a41-8e39-4d16-ae87-0e030773dc09-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:51.638026 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.637906 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqt7m\" (UniqueName: \"kubernetes.io/projected/d3ea1a41-8e39-4d16-ae87-0e030773dc09-kube-api-access-bqt7m\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:51.730420 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.730386 5258 generic.go:356] "Generic (PLEG): container finished" podID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerID="8ae33980838366191d5dbd5fa627365bd0b36e72cd72aecb95467135682de5eb" exitCode=0 Nov 04 05:28:51.730605 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.730503 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3ea1a41-8e39-4d16-ae87-0e030773dc09","Type":"ContainerDied","Data":"8ae33980838366191d5dbd5fa627365bd0b36e72cd72aecb95467135682de5eb"} Nov 04 05:28:51.730605 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.730560 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d3ea1a41-8e39-4d16-ae87-0e030773dc09","Type":"ContainerDied","Data":"5db860688e5dd154c8977c208ee2f94578f3c26bba2b40fb148efa4cd6e99790"} Nov 04 05:28:51.730605 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.730594 5258 scope.go:122] "RemoveContainer" containerID="91e78c938b656502c5de651aa3a03e936abb1ff70d36673caef95f20f37afe7b" Nov 04 05:28:51.730786 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.730757 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 05:28:51.740514 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.740496 5258 scope.go:122] "RemoveContainer" containerID="5645f1cbfc23dbf7f1df2d6f1b25d59fff57826b88df18717240801d7f55b48a" Nov 04 05:28:51.748878 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.748852 5258 scope.go:122] "RemoveContainer" containerID="3303030c71badc37f6cac49fb374fc7a0522a4b1557d3f3def975b0b5155a0f6" Nov 04 05:28:51.756409 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.756383 5258 scope.go:122] "RemoveContainer" containerID="8ae33980838366191d5dbd5fa627365bd0b36e72cd72aecb95467135682de5eb" Nov 04 05:28:51.763675 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.763654 5258 scope.go:122] "RemoveContainer" containerID="75bd935f2e20d910c08be90e3209141921bdf4f0b90d06593296a5af6295f0b0" Nov 04 05:28:51.771235 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.771202 5258 scope.go:122] "RemoveContainer" containerID="64848ccfcbea01b7a469debaf158fc72758eadd1a01036c1ebb288daea728e6d" Nov 04 05:28:51.778146 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.778123 5258 scope.go:122] "RemoveContainer" containerID="1d221231c4d9e99a74ebc078aab66afda3ab84ebe52355ceec8651bde41c3f98" Nov 04 05:28:51.786494 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.786452 5258 scope.go:122] "RemoveContainer" containerID="91e78c938b656502c5de651aa3a03e936abb1ff70d36673caef95f20f37afe7b" Nov 04 05:28:51.786728 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:28:51.786708 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91e78c938b656502c5de651aa3a03e936abb1ff70d36673caef95f20f37afe7b\": container with ID starting with 91e78c938b656502c5de651aa3a03e936abb1ff70d36673caef95f20f37afe7b not found: ID does not exist" containerID="91e78c938b656502c5de651aa3a03e936abb1ff70d36673caef95f20f37afe7b" Nov 04 05:28:51.786776 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.786741 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91e78c938b656502c5de651aa3a03e936abb1ff70d36673caef95f20f37afe7b"} err="failed to get container status \"91e78c938b656502c5de651aa3a03e936abb1ff70d36673caef95f20f37afe7b\": rpc error: code = NotFound desc = could not find container \"91e78c938b656502c5de651aa3a03e936abb1ff70d36673caef95f20f37afe7b\": container with ID starting with 91e78c938b656502c5de651aa3a03e936abb1ff70d36673caef95f20f37afe7b not found: ID does not exist" Nov 04 05:28:51.786776 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.786759 5258 scope.go:122] "RemoveContainer" containerID="5645f1cbfc23dbf7f1df2d6f1b25d59fff57826b88df18717240801d7f55b48a" Nov 04 05:28:51.786980 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:28:51.786967 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5645f1cbfc23dbf7f1df2d6f1b25d59fff57826b88df18717240801d7f55b48a\": container with ID starting with 5645f1cbfc23dbf7f1df2d6f1b25d59fff57826b88df18717240801d7f55b48a not found: ID does not exist" containerID="5645f1cbfc23dbf7f1df2d6f1b25d59fff57826b88df18717240801d7f55b48a" Nov 04 05:28:51.787026 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.786983 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5645f1cbfc23dbf7f1df2d6f1b25d59fff57826b88df18717240801d7f55b48a"} err="failed to get container status \"5645f1cbfc23dbf7f1df2d6f1b25d59fff57826b88df18717240801d7f55b48a\": rpc error: code = NotFound desc = could not find container \"5645f1cbfc23dbf7f1df2d6f1b25d59fff57826b88df18717240801d7f55b48a\": container with ID starting with 5645f1cbfc23dbf7f1df2d6f1b25d59fff57826b88df18717240801d7f55b48a not found: ID does not exist" Nov 04 05:28:51.787026 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.786995 5258 scope.go:122] "RemoveContainer" containerID="3303030c71badc37f6cac49fb374fc7a0522a4b1557d3f3def975b0b5155a0f6" Nov 04 05:28:51.787231 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:28:51.787213 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3303030c71badc37f6cac49fb374fc7a0522a4b1557d3f3def975b0b5155a0f6\": container with ID starting with 3303030c71badc37f6cac49fb374fc7a0522a4b1557d3f3def975b0b5155a0f6 not found: ID does not exist" containerID="3303030c71badc37f6cac49fb374fc7a0522a4b1557d3f3def975b0b5155a0f6" Nov 04 05:28:51.787273 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.787237 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3303030c71badc37f6cac49fb374fc7a0522a4b1557d3f3def975b0b5155a0f6"} err="failed to get container status \"3303030c71badc37f6cac49fb374fc7a0522a4b1557d3f3def975b0b5155a0f6\": rpc error: code = NotFound desc = could not find container \"3303030c71badc37f6cac49fb374fc7a0522a4b1557d3f3def975b0b5155a0f6\": container with ID starting with 3303030c71badc37f6cac49fb374fc7a0522a4b1557d3f3def975b0b5155a0f6 not found: ID does not exist" Nov 04 05:28:51.787273 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.787252 5258 scope.go:122] "RemoveContainer" containerID="8ae33980838366191d5dbd5fa627365bd0b36e72cd72aecb95467135682de5eb" Nov 04 05:28:51.787496 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:28:51.787476 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ae33980838366191d5dbd5fa627365bd0b36e72cd72aecb95467135682de5eb\": container with ID starting with 8ae33980838366191d5dbd5fa627365bd0b36e72cd72aecb95467135682de5eb not found: ID does not exist" containerID="8ae33980838366191d5dbd5fa627365bd0b36e72cd72aecb95467135682de5eb" Nov 04 05:28:51.787561 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.787506 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ae33980838366191d5dbd5fa627365bd0b36e72cd72aecb95467135682de5eb"} err="failed to get container status \"8ae33980838366191d5dbd5fa627365bd0b36e72cd72aecb95467135682de5eb\": rpc error: code = NotFound desc = could not find container \"8ae33980838366191d5dbd5fa627365bd0b36e72cd72aecb95467135682de5eb\": container with ID starting with 8ae33980838366191d5dbd5fa627365bd0b36e72cd72aecb95467135682de5eb not found: ID does not exist" Nov 04 05:28:51.787561 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.787527 5258 scope.go:122] "RemoveContainer" containerID="75bd935f2e20d910c08be90e3209141921bdf4f0b90d06593296a5af6295f0b0" Nov 04 05:28:51.787757 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:28:51.787741 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75bd935f2e20d910c08be90e3209141921bdf4f0b90d06593296a5af6295f0b0\": container with ID starting with 75bd935f2e20d910c08be90e3209141921bdf4f0b90d06593296a5af6295f0b0 not found: ID does not exist" containerID="75bd935f2e20d910c08be90e3209141921bdf4f0b90d06593296a5af6295f0b0" Nov 04 05:28:51.787793 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.787761 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75bd935f2e20d910c08be90e3209141921bdf4f0b90d06593296a5af6295f0b0"} err="failed to get container status \"75bd935f2e20d910c08be90e3209141921bdf4f0b90d06593296a5af6295f0b0\": rpc error: code = NotFound desc = could not find container \"75bd935f2e20d910c08be90e3209141921bdf4f0b90d06593296a5af6295f0b0\": container with ID starting with 75bd935f2e20d910c08be90e3209141921bdf4f0b90d06593296a5af6295f0b0 not found: ID does not exist" Nov 04 05:28:51.787793 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.787774 5258 scope.go:122] "RemoveContainer" containerID="64848ccfcbea01b7a469debaf158fc72758eadd1a01036c1ebb288daea728e6d" Nov 04 05:28:51.787965 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:28:51.787948 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64848ccfcbea01b7a469debaf158fc72758eadd1a01036c1ebb288daea728e6d\": container with ID starting with 64848ccfcbea01b7a469debaf158fc72758eadd1a01036c1ebb288daea728e6d not found: ID does not exist" containerID="64848ccfcbea01b7a469debaf158fc72758eadd1a01036c1ebb288daea728e6d" Nov 04 05:28:51.788028 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.787973 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64848ccfcbea01b7a469debaf158fc72758eadd1a01036c1ebb288daea728e6d"} err="failed to get container status \"64848ccfcbea01b7a469debaf158fc72758eadd1a01036c1ebb288daea728e6d\": rpc error: code = NotFound desc = could not find container \"64848ccfcbea01b7a469debaf158fc72758eadd1a01036c1ebb288daea728e6d\": container with ID starting with 64848ccfcbea01b7a469debaf158fc72758eadd1a01036c1ebb288daea728e6d not found: ID does not exist" Nov 04 05:28:51.788028 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.787994 5258 scope.go:122] "RemoveContainer" containerID="1d221231c4d9e99a74ebc078aab66afda3ab84ebe52355ceec8651bde41c3f98" Nov 04 05:28:51.788272 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:28:51.788254 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d221231c4d9e99a74ebc078aab66afda3ab84ebe52355ceec8651bde41c3f98\": container with ID starting with 1d221231c4d9e99a74ebc078aab66afda3ab84ebe52355ceec8651bde41c3f98 not found: ID does not exist" containerID="1d221231c4d9e99a74ebc078aab66afda3ab84ebe52355ceec8651bde41c3f98" Nov 04 05:28:51.788357 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.788274 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d221231c4d9e99a74ebc078aab66afda3ab84ebe52355ceec8651bde41c3f98"} err="failed to get container status \"1d221231c4d9e99a74ebc078aab66afda3ab84ebe52355ceec8651bde41c3f98\": rpc error: code = NotFound desc = could not find container \"1d221231c4d9e99a74ebc078aab66afda3ab84ebe52355ceec8651bde41c3f98\": container with ID starting with 1d221231c4d9e99a74ebc078aab66afda3ab84ebe52355ceec8651bde41c3f98 not found: ID does not exist" Nov 04 05:28:51.891889 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.891849 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 05:28:51.896428 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:51.896401 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 05:28:52.268241 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:52.268212 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" path="/var/lib/kubelet/pods/d3ea1a41-8e39-4d16-ae87-0e030773dc09/volumes" Nov 04 05:28:52.779719 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:28:52.779684 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:28:55.965260 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:55.965223 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 05:28:55.965794 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:55.965492 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="prometheus" containerID="cri-o://85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526" gracePeriod=600 Nov 04 05:28:55.965794 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:55.965537 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="thanos-sidecar" containerID="cri-o://96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554" gracePeriod=600 Nov 04 05:28:55.965794 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:55.965563 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="kube-rbac-proxy" containerID="cri-o://5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7" gracePeriod=600 Nov 04 05:28:55.965794 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:55.965575 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="kube-rbac-proxy-web" containerID="cri-o://f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6" gracePeriod=600 Nov 04 05:28:55.965794 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:55.965586 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="config-reloader" containerID="cri-o://63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294" gracePeriod=600 Nov 04 05:28:55.965794 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:55.965588 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="kube-rbac-proxy-thanos" containerID="cri-o://8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d" gracePeriod=600 Nov 04 05:28:56.204978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.204955 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.277711 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.277640 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-configmap-kubelet-serving-ca-bundle\") pod \"e42acc16-8c50-4990-8f7e-35ac3df287f2\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " Nov 04 05:28:56.277711 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.277685 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-configmap-serving-certs-ca-bundle\") pod \"e42acc16-8c50-4990-8f7e-35ac3df287f2\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " Nov 04 05:28:56.277891 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.277731 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-config\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-config\") pod \"e42acc16-8c50-4990-8f7e-35ac3df287f2\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " Nov 04 05:28:56.277891 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.277758 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-thanos-prometheus-http-client-file\") pod \"e42acc16-8c50-4990-8f7e-35ac3df287f2\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " Nov 04 05:28:56.277891 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.277778 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-web-config\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-web-config\") pod \"e42acc16-8c50-4990-8f7e-35ac3df287f2\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " Nov 04 05:28:56.277891 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.277795 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-rulefiles-2\") pod \"e42acc16-8c50-4990-8f7e-35ac3df287f2\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " Nov 04 05:28:56.277891 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.277823 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-prometheus-k8s-tls\") pod \"e42acc16-8c50-4990-8f7e-35ac3df287f2\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " Nov 04 05:28:56.277891 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.277863 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e42acc16-8c50-4990-8f7e-35ac3df287f2-config-out\" (UniqueName: \"kubernetes.io/empty-dir/e42acc16-8c50-4990-8f7e-35ac3df287f2-config-out\") pod \"e42acc16-8c50-4990-8f7e-35ac3df287f2\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " Nov 04 05:28:56.278219 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.277907 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e42acc16-8c50-4990-8f7e-35ac3df287f2-tls-assets\" (UniqueName: \"kubernetes.io/projected/e42acc16-8c50-4990-8f7e-35ac3df287f2-tls-assets\") pod \"e42acc16-8c50-4990-8f7e-35ac3df287f2\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " Nov 04 05:28:56.278219 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.277955 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-trusted-ca-bundle\") pod \"e42acc16-8c50-4990-8f7e-35ac3df287f2\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " Nov 04 05:28:56.278219 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.277982 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e42acc16-8c50-4990-8f7e-35ac3df287f2\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " Nov 04 05:28:56.278219 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.278019 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-grpc-tls\") pod \"e42acc16-8c50-4990-8f7e-35ac3df287f2\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " Nov 04 05:28:56.278219 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.278080 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-configmap-metrics-client-ca\") pod \"e42acc16-8c50-4990-8f7e-35ac3df287f2\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " Nov 04 05:28:56.278219 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.278108 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-kube-rbac-proxy\") pod \"e42acc16-8c50-4990-8f7e-35ac3df287f2\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " Nov 04 05:28:56.278219 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.278144 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-rulefiles-0\") pod \"e42acc16-8c50-4990-8f7e-35ac3df287f2\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " Nov 04 05:28:56.278219 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.278157 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-configmap-kubelet-serving-ca-bundle" pod "e42acc16-8c50-4990-8f7e-35ac3df287f2" (UID: "e42acc16-8c50-4990-8f7e-35ac3df287f2"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:56.278219 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.278190 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-db\") pod \"e42acc16-8c50-4990-8f7e-35ac3df287f2\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " Nov 04 05:28:56.278670 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.278239 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e42acc16-8c50-4990-8f7e-35ac3df287f2\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " Nov 04 05:28:56.278670 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.278272 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-metrics-client-certs\") pod \"e42acc16-8c50-4990-8f7e-35ac3df287f2\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " Nov 04 05:28:56.278670 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.278271 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-rulefiles-2" pod "e42acc16-8c50-4990-8f7e-35ac3df287f2" (UID: "e42acc16-8c50-4990-8f7e-35ac3df287f2"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:56.278670 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.278326 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-rulefiles-1\") pod \"e42acc16-8c50-4990-8f7e-35ac3df287f2\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " Nov 04 05:28:56.278670 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.278356 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e42acc16-8c50-4990-8f7e-35ac3df287f2-kube-api-access-84b76\" (UniqueName: \"kubernetes.io/projected/e42acc16-8c50-4990-8f7e-35ac3df287f2-kube-api-access-84b76\") pod \"e42acc16-8c50-4990-8f7e-35ac3df287f2\" (UID: \"e42acc16-8c50-4990-8f7e-35ac3df287f2\") " Nov 04 05:28:56.278670 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.278595 5258 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:56.278670 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.278614 5258 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:56.279376 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.279084 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-configmap-serving-certs-ca-bundle" pod "e42acc16-8c50-4990-8f7e-35ac3df287f2" (UID: "e42acc16-8c50-4990-8f7e-35ac3df287f2"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:56.280148 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.280088 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-rulefiles-0" pod "e42acc16-8c50-4990-8f7e-35ac3df287f2" (UID: "e42acc16-8c50-4990-8f7e-35ac3df287f2"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:56.280148 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.280101 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-db" pod "e42acc16-8c50-4990-8f7e-35ac3df287f2" (UID: "e42acc16-8c50-4990-8f7e-35ac3df287f2"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:28:56.280903 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.280571 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-trusted-ca-bundle" pod "e42acc16-8c50-4990-8f7e-35ac3df287f2" (UID: "e42acc16-8c50-4990-8f7e-35ac3df287f2"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:56.280903 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.280661 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-rulefiles-1" pod "e42acc16-8c50-4990-8f7e-35ac3df287f2" (UID: "e42acc16-8c50-4990-8f7e-35ac3df287f2"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:56.281087 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.280926 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-prometheus-k8s-tls" pod "e42acc16-8c50-4990-8f7e-35ac3df287f2" (UID: "e42acc16-8c50-4990-8f7e-35ac3df287f2"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:56.281087 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.281054 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-configmap-metrics-client-ca" pod "e42acc16-8c50-4990-8f7e-35ac3df287f2" (UID: "e42acc16-8c50-4990-8f7e-35ac3df287f2"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:28:56.281343 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.281282 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-config" pod "e42acc16-8c50-4990-8f7e-35ac3df287f2" (UID: "e42acc16-8c50-4990-8f7e-35ac3df287f2"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:56.281343 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.281303 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-thanos-prometheus-http-client-file" pod "e42acc16-8c50-4990-8f7e-35ac3df287f2" (UID: "e42acc16-8c50-4990-8f7e-35ac3df287f2"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:56.281489 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.281378 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-prometheus-k8s-kube-rbac-proxy-web" pod "e42acc16-8c50-4990-8f7e-35ac3df287f2" (UID: "e42acc16-8c50-4990-8f7e-35ac3df287f2"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:56.281598 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.281571 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-grpc-tls" pod "e42acc16-8c50-4990-8f7e-35ac3df287f2" (UID: "e42acc16-8c50-4990-8f7e-35ac3df287f2"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:56.281844 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.281808 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e42acc16-8c50-4990-8f7e-35ac3df287f2-kube-api-access-84b76" pod "e42acc16-8c50-4990-8f7e-35ac3df287f2" (UID: "e42acc16-8c50-4990-8f7e-35ac3df287f2"). InnerVolumeSpecName "kube-api-access-84b76". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:28:56.282361 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.282339 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-kube-rbac-proxy" pod "e42acc16-8c50-4990-8f7e-35ac3df287f2" (UID: "e42acc16-8c50-4990-8f7e-35ac3df287f2"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:56.282526 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.282510 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e42acc16-8c50-4990-8f7e-35ac3df287f2-config-out" pod "e42acc16-8c50-4990-8f7e-35ac3df287f2" (UID: "e42acc16-8c50-4990-8f7e-35ac3df287f2"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:28:56.282858 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.282833 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-metrics-client-certs" pod "e42acc16-8c50-4990-8f7e-35ac3df287f2" (UID: "e42acc16-8c50-4990-8f7e-35ac3df287f2"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:56.283518 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.283489 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e42acc16-8c50-4990-8f7e-35ac3df287f2-tls-assets" pod "e42acc16-8c50-4990-8f7e-35ac3df287f2" (UID: "e42acc16-8c50-4990-8f7e-35ac3df287f2"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:28:56.283767 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.283744 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-prometheus-k8s-thanos-sidecar-tls" pod "e42acc16-8c50-4990-8f7e-35ac3df287f2" (UID: "e42acc16-8c50-4990-8f7e-35ac3df287f2"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:56.291767 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.291746 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-web-config" pod "e42acc16-8c50-4990-8f7e-35ac3df287f2" (UID: "e42acc16-8c50-4990-8f7e-35ac3df287f2"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:28:56.379271 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.379229 5258 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:56.379271 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.379257 5258 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e42acc16-8c50-4990-8f7e-35ac3df287f2-config-out\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:56.379271 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.379266 5258 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e42acc16-8c50-4990-8f7e-35ac3df287f2-tls-assets\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:56.379271 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.379276 5258 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:56.379632 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.379290 5258 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:56.379632 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.379299 5258 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-grpc-tls\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:56.379632 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.379307 5258 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-configmap-metrics-client-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:56.379632 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.379318 5258 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-kube-rbac-proxy\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:56.379632 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.379328 5258 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:56.379632 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.379337 5258 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-db\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:56.379632 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.379345 5258 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:56.379632 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.379357 5258 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-secret-metrics-client-certs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:56.379632 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.379372 5258 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:56.379632 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.379380 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84b76\" (UniqueName: \"kubernetes.io/projected/e42acc16-8c50-4990-8f7e-35ac3df287f2-kube-api-access-84b76\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:56.379632 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.379390 5258 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e42acc16-8c50-4990-8f7e-35ac3df287f2-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:56.379632 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.379399 5258 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-config\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:56.379632 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.379407 5258 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:56.379632 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.379416 5258 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e42acc16-8c50-4990-8f7e-35ac3df287f2-web-config\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:28:56.753314 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.753279 5258 generic.go:356] "Generic (PLEG): container finished" podID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerID="8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d" exitCode=0 Nov 04 05:28:56.753314 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.753306 5258 generic.go:356] "Generic (PLEG): container finished" podID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerID="5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7" exitCode=0 Nov 04 05:28:56.753314 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.753313 5258 generic.go:356] "Generic (PLEG): container finished" podID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerID="f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6" exitCode=0 Nov 04 05:28:56.753314 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.753318 5258 generic.go:356] "Generic (PLEG): container finished" podID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerID="96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554" exitCode=0 Nov 04 05:28:56.753314 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.753324 5258 generic.go:356] "Generic (PLEG): container finished" podID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerID="63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294" exitCode=0 Nov 04 05:28:56.753747 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.753330 5258 generic.go:356] "Generic (PLEG): container finished" podID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerID="85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526" exitCode=0 Nov 04 05:28:56.753747 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.753375 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:56.753747 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.753372 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e42acc16-8c50-4990-8f7e-35ac3df287f2","Type":"ContainerDied","Data":"8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d"} Nov 04 05:28:56.753747 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.753487 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e42acc16-8c50-4990-8f7e-35ac3df287f2","Type":"ContainerDied","Data":"5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7"} Nov 04 05:28:56.753747 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.753533 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e42acc16-8c50-4990-8f7e-35ac3df287f2","Type":"ContainerDied","Data":"f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6"} Nov 04 05:28:56.753747 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.753549 5258 scope.go:122] "RemoveContainer" containerID="8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d" Nov 04 05:28:56.753747 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.753576 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e42acc16-8c50-4990-8f7e-35ac3df287f2","Type":"ContainerDied","Data":"96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554"} Nov 04 05:28:56.753747 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.753618 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e42acc16-8c50-4990-8f7e-35ac3df287f2","Type":"ContainerDied","Data":"63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294"} Nov 04 05:28:56.753747 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.753661 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e42acc16-8c50-4990-8f7e-35ac3df287f2","Type":"ContainerDied","Data":"85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526"} Nov 04 05:28:56.753747 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.753701 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e42acc16-8c50-4990-8f7e-35ac3df287f2","Type":"ContainerDied","Data":"30117051ab1899940396e42201ab0a4de429c3ae79bbe5164e134aaf76dd3660"} Nov 04 05:28:56.761495 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.761474 5258 scope.go:122] "RemoveContainer" containerID="5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7" Nov 04 05:28:56.768136 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.768119 5258 scope.go:122] "RemoveContainer" containerID="f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6" Nov 04 05:28:56.774359 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.774337 5258 scope.go:122] "RemoveContainer" containerID="96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554" Nov 04 05:28:56.782502 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.782486 5258 scope.go:122] "RemoveContainer" containerID="63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294" Nov 04 05:28:56.788847 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.788831 5258 scope.go:122] "RemoveContainer" containerID="85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526" Nov 04 05:28:56.795817 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.795800 5258 scope.go:122] "RemoveContainer" containerID="38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31" Nov 04 05:28:56.801953 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.801939 5258 scope.go:122] "RemoveContainer" containerID="8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d" Nov 04 05:28:56.802217 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:28:56.802199 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d\": container with ID starting with 8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d not found: ID does not exist" containerID="8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d" Nov 04 05:28:56.802267 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.802224 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d"} err="failed to get container status \"8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d\": rpc error: code = NotFound desc = could not find container \"8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d\": container with ID starting with 8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d not found: ID does not exist" Nov 04 05:28:56.802267 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.802242 5258 scope.go:122] "RemoveContainer" containerID="5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7" Nov 04 05:28:56.802470 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:28:56.802450 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7\": container with ID starting with 5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7 not found: ID does not exist" containerID="5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7" Nov 04 05:28:56.802509 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.802477 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7"} err="failed to get container status \"5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7\": rpc error: code = NotFound desc = could not find container \"5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7\": container with ID starting with 5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7 not found: ID does not exist" Nov 04 05:28:56.802509 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.802494 5258 scope.go:122] "RemoveContainer" containerID="f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6" Nov 04 05:28:56.802743 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:28:56.802725 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6\": container with ID starting with f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6 not found: ID does not exist" containerID="f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6" Nov 04 05:28:56.802777 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.802748 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6"} err="failed to get container status \"f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6\": rpc error: code = NotFound desc = could not find container \"f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6\": container with ID starting with f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6 not found: ID does not exist" Nov 04 05:28:56.802777 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.802764 5258 scope.go:122] "RemoveContainer" containerID="96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554" Nov 04 05:28:56.802977 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:28:56.802959 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554\": container with ID starting with 96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554 not found: ID does not exist" containerID="96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554" Nov 04 05:28:56.803020 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.802982 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554"} err="failed to get container status \"96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554\": rpc error: code = NotFound desc = could not find container \"96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554\": container with ID starting with 96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554 not found: ID does not exist" Nov 04 05:28:56.803020 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.802994 5258 scope.go:122] "RemoveContainer" containerID="63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294" Nov 04 05:28:56.803236 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:28:56.803222 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294\": container with ID starting with 63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294 not found: ID does not exist" containerID="63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294" Nov 04 05:28:56.803289 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.803240 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294"} err="failed to get container status \"63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294\": rpc error: code = NotFound desc = could not find container \"63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294\": container with ID starting with 63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294 not found: ID does not exist" Nov 04 05:28:56.803289 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.803251 5258 scope.go:122] "RemoveContainer" containerID="85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526" Nov 04 05:28:56.803497 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:28:56.803479 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526\": container with ID starting with 85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526 not found: ID does not exist" containerID="85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526" Nov 04 05:28:56.803593 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.803502 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526"} err="failed to get container status \"85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526\": rpc error: code = NotFound desc = could not find container \"85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526\": container with ID starting with 85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526 not found: ID does not exist" Nov 04 05:28:56.803593 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.803519 5258 scope.go:122] "RemoveContainer" containerID="38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31" Nov 04 05:28:56.803805 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:28:56.803786 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31\": container with ID starting with 38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31 not found: ID does not exist" containerID="38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31" Nov 04 05:28:56.803865 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.803815 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31"} err="failed to get container status \"38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31\": rpc error: code = NotFound desc = could not find container \"38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31\": container with ID starting with 38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31 not found: ID does not exist" Nov 04 05:28:56.803865 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.803831 5258 scope.go:122] "RemoveContainer" containerID="8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d" Nov 04 05:28:56.804075 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.804028 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d"} err="failed to get container status \"8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d\": rpc error: code = NotFound desc = could not find container \"8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d\": container with ID starting with 8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d not found: ID does not exist" Nov 04 05:28:56.804075 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.804074 5258 scope.go:122] "RemoveContainer" containerID="5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7" Nov 04 05:28:56.804278 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.804261 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7"} err="failed to get container status \"5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7\": rpc error: code = NotFound desc = could not find container \"5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7\": container with ID starting with 5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7 not found: ID does not exist" Nov 04 05:28:56.804332 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.804278 5258 scope.go:122] "RemoveContainer" containerID="f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6" Nov 04 05:28:56.804463 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.804447 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6"} err="failed to get container status \"f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6\": rpc error: code = NotFound desc = could not find container \"f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6\": container with ID starting with f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6 not found: ID does not exist" Nov 04 05:28:56.804526 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.804462 5258 scope.go:122] "RemoveContainer" containerID="96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554" Nov 04 05:28:56.804659 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.804639 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554"} err="failed to get container status \"96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554\": rpc error: code = NotFound desc = could not find container \"96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554\": container with ID starting with 96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554 not found: ID does not exist" Nov 04 05:28:56.804723 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.804660 5258 scope.go:122] "RemoveContainer" containerID="63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294" Nov 04 05:28:56.804853 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.804838 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294"} err="failed to get container status \"63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294\": rpc error: code = NotFound desc = could not find container \"63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294\": container with ID starting with 63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294 not found: ID does not exist" Nov 04 05:28:56.804853 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.804851 5258 scope.go:122] "RemoveContainer" containerID="85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526" Nov 04 05:28:56.805059 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.805028 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526"} err="failed to get container status \"85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526\": rpc error: code = NotFound desc = could not find container \"85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526\": container with ID starting with 85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526 not found: ID does not exist" Nov 04 05:28:56.805260 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.805060 5258 scope.go:122] "RemoveContainer" containerID="38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31" Nov 04 05:28:56.805317 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.805256 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31"} err="failed to get container status \"38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31\": rpc error: code = NotFound desc = could not find container \"38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31\": container with ID starting with 38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31 not found: ID does not exist" Nov 04 05:28:56.805317 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.805269 5258 scope.go:122] "RemoveContainer" containerID="8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d" Nov 04 05:28:56.805470 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.805451 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d"} err="failed to get container status \"8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d\": rpc error: code = NotFound desc = could not find container \"8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d\": container with ID starting with 8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d not found: ID does not exist" Nov 04 05:28:56.805470 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.805468 5258 scope.go:122] "RemoveContainer" containerID="5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7" Nov 04 05:28:56.805661 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.805641 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7"} err="failed to get container status \"5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7\": rpc error: code = NotFound desc = could not find container \"5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7\": container with ID starting with 5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7 not found: ID does not exist" Nov 04 05:28:56.805661 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.805661 5258 scope.go:122] "RemoveContainer" containerID="f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6" Nov 04 05:28:56.805882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.805862 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6"} err="failed to get container status \"f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6\": rpc error: code = NotFound desc = could not find container \"f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6\": container with ID starting with f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6 not found: ID does not exist" Nov 04 05:28:56.805928 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.805884 5258 scope.go:122] "RemoveContainer" containerID="96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554" Nov 04 05:28:56.806104 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.806084 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554"} err="failed to get container status \"96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554\": rpc error: code = NotFound desc = could not find container \"96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554\": container with ID starting with 96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554 not found: ID does not exist" Nov 04 05:28:56.806156 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.806105 5258 scope.go:122] "RemoveContainer" containerID="63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294" Nov 04 05:28:56.806292 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.806276 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294"} err="failed to get container status \"63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294\": rpc error: code = NotFound desc = could not find container \"63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294\": container with ID starting with 63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294 not found: ID does not exist" Nov 04 05:28:56.806333 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.806292 5258 scope.go:122] "RemoveContainer" containerID="85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526" Nov 04 05:28:56.806495 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.806476 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526"} err="failed to get container status \"85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526\": rpc error: code = NotFound desc = could not find container \"85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526\": container with ID starting with 85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526 not found: ID does not exist" Nov 04 05:28:56.806561 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.806497 5258 scope.go:122] "RemoveContainer" containerID="38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31" Nov 04 05:28:56.806701 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.806686 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31"} err="failed to get container status \"38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31\": rpc error: code = NotFound desc = could not find container \"38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31\": container with ID starting with 38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31 not found: ID does not exist" Nov 04 05:28:56.806748 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.806701 5258 scope.go:122] "RemoveContainer" containerID="8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d" Nov 04 05:28:56.806908 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.806894 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d"} err="failed to get container status \"8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d\": rpc error: code = NotFound desc = could not find container \"8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d\": container with ID starting with 8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d not found: ID does not exist" Nov 04 05:28:56.806908 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.806907 5258 scope.go:122] "RemoveContainer" containerID="5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7" Nov 04 05:28:56.807124 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.807107 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7"} err="failed to get container status \"5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7\": rpc error: code = NotFound desc = could not find container \"5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7\": container with ID starting with 5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7 not found: ID does not exist" Nov 04 05:28:56.807165 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.807125 5258 scope.go:122] "RemoveContainer" containerID="f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6" Nov 04 05:28:56.807328 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.807311 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6"} err="failed to get container status \"f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6\": rpc error: code = NotFound desc = could not find container \"f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6\": container with ID starting with f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6 not found: ID does not exist" Nov 04 05:28:56.807403 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.807331 5258 scope.go:122] "RemoveContainer" containerID="96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554" Nov 04 05:28:56.807543 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.807527 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554"} err="failed to get container status \"96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554\": rpc error: code = NotFound desc = could not find container \"96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554\": container with ID starting with 96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554 not found: ID does not exist" Nov 04 05:28:56.807587 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.807543 5258 scope.go:122] "RemoveContainer" containerID="63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294" Nov 04 05:28:56.807714 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.807699 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294"} err="failed to get container status \"63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294\": rpc error: code = NotFound desc = could not find container \"63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294\": container with ID starting with 63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294 not found: ID does not exist" Nov 04 05:28:56.807755 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.807714 5258 scope.go:122] "RemoveContainer" containerID="85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526" Nov 04 05:28:56.807896 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.807880 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526"} err="failed to get container status \"85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526\": rpc error: code = NotFound desc = could not find container \"85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526\": container with ID starting with 85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526 not found: ID does not exist" Nov 04 05:28:56.807939 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.807895 5258 scope.go:122] "RemoveContainer" containerID="38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31" Nov 04 05:28:56.808089 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.808073 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31"} err="failed to get container status \"38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31\": rpc error: code = NotFound desc = could not find container \"38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31\": container with ID starting with 38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31 not found: ID does not exist" Nov 04 05:28:56.808089 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.808089 5258 scope.go:122] "RemoveContainer" containerID="8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d" Nov 04 05:28:56.808284 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.808259 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d"} err="failed to get container status \"8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d\": rpc error: code = NotFound desc = could not find container \"8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d\": container with ID starting with 8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d not found: ID does not exist" Nov 04 05:28:56.808320 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.808284 5258 scope.go:122] "RemoveContainer" containerID="5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7" Nov 04 05:28:56.808461 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.808447 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7"} err="failed to get container status \"5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7\": rpc error: code = NotFound desc = could not find container \"5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7\": container with ID starting with 5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7 not found: ID does not exist" Nov 04 05:28:56.808501 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.808461 5258 scope.go:122] "RemoveContainer" containerID="f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6" Nov 04 05:28:56.808636 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.808622 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6"} err="failed to get container status \"f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6\": rpc error: code = NotFound desc = could not find container \"f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6\": container with ID starting with f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6 not found: ID does not exist" Nov 04 05:28:56.808676 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.808636 5258 scope.go:122] "RemoveContainer" containerID="96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554" Nov 04 05:28:56.808812 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.808796 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554"} err="failed to get container status \"96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554\": rpc error: code = NotFound desc = could not find container \"96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554\": container with ID starting with 96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554 not found: ID does not exist" Nov 04 05:28:56.808851 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.808811 5258 scope.go:122] "RemoveContainer" containerID="63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294" Nov 04 05:28:56.809013 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.808980 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294"} err="failed to get container status \"63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294\": rpc error: code = NotFound desc = could not find container \"63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294\": container with ID starting with 63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294 not found: ID does not exist" Nov 04 05:28:56.809102 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.809014 5258 scope.go:122] "RemoveContainer" containerID="85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526" Nov 04 05:28:56.809221 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.809207 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526"} err="failed to get container status \"85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526\": rpc error: code = NotFound desc = could not find container \"85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526\": container with ID starting with 85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526 not found: ID does not exist" Nov 04 05:28:56.809261 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.809221 5258 scope.go:122] "RemoveContainer" containerID="38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31" Nov 04 05:28:56.809437 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.809420 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31"} err="failed to get container status \"38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31\": rpc error: code = NotFound desc = could not find container \"38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31\": container with ID starting with 38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31 not found: ID does not exist" Nov 04 05:28:56.809529 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.809438 5258 scope.go:122] "RemoveContainer" containerID="8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d" Nov 04 05:28:56.809616 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.809602 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d"} err="failed to get container status \"8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d\": rpc error: code = NotFound desc = could not find container \"8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d\": container with ID starting with 8470505309f6ef80dab5fab79e017ca7d7b5441c1371e4c3c0d65a715225ee1d not found: ID does not exist" Nov 04 05:28:56.809654 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.809616 5258 scope.go:122] "RemoveContainer" containerID="5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7" Nov 04 05:28:56.809786 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.809772 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7"} err="failed to get container status \"5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7\": rpc error: code = NotFound desc = could not find container \"5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7\": container with ID starting with 5791485bb7a48e66611bba622e0235cfcbbc7586c66fe08f25988066e59165b7 not found: ID does not exist" Nov 04 05:28:56.809833 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.809786 5258 scope.go:122] "RemoveContainer" containerID="f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6" Nov 04 05:28:56.809967 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.809951 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6"} err="failed to get container status \"f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6\": rpc error: code = NotFound desc = could not find container \"f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6\": container with ID starting with f4da762d467e5858dc14edf7d3aab0edbc46bb2be1be5436bcec70c27c2479f6 not found: ID does not exist" Nov 04 05:28:56.810010 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.809968 5258 scope.go:122] "RemoveContainer" containerID="96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554" Nov 04 05:28:56.810220 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.810199 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554"} err="failed to get container status \"96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554\": rpc error: code = NotFound desc = could not find container \"96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554\": container with ID starting with 96b251a5f3c448dcb8b893ea2431ecd9a40a115ec869672b31ae25318a31e554 not found: ID does not exist" Nov 04 05:28:56.810220 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.810218 5258 scope.go:122] "RemoveContainer" containerID="63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294" Nov 04 05:28:56.810435 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.810416 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294"} err="failed to get container status \"63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294\": rpc error: code = NotFound desc = could not find container \"63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294\": container with ID starting with 63e53637c6c0f075a3a3223a06673686685c73caac1bf067e2d495a807f8d294 not found: ID does not exist" Nov 04 05:28:56.810479 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.810436 5258 scope.go:122] "RemoveContainer" containerID="85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526" Nov 04 05:28:56.810645 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.810629 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526"} err="failed to get container status \"85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526\": rpc error: code = NotFound desc = could not find container \"85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526\": container with ID starting with 85dbb6321b9472831bd300a5f8d23e8fadc14b92710dfef7db456e84a95a0526 not found: ID does not exist" Nov 04 05:28:56.810687 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.810645 5258 scope.go:122] "RemoveContainer" containerID="38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31" Nov 04 05:28:56.810849 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.810827 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31"} err="failed to get container status \"38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31\": rpc error: code = NotFound desc = could not find container \"38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31\": container with ID starting with 38e4b9cdb1a138a0ed1a27a9aad5cdebf5f2dd761ffce2cdc00ebb83c6ddfc31 not found: ID does not exist" Nov 04 05:28:56.919234 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.919206 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 05:28:56.966168 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.966143 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 05:28:56.966698 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.966681 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 05:28:56.966951 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.966939 5258 cpu_manager.go:414] "containerMap: removing container" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="config-reloader" Nov 04 05:28:56.966991 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.966952 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="config-reloader" Nov 04 05:28:56.966991 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.966963 5258 cpu_manager.go:414] "containerMap: removing container" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="config-reloader" Nov 04 05:28:56.966991 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.966968 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="config-reloader" Nov 04 05:28:56.966991 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.966975 5258 cpu_manager.go:414] "containerMap: removing container" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="kube-rbac-proxy" Nov 04 05:28:56.966991 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.966980 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="kube-rbac-proxy" Nov 04 05:28:56.966991 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.966986 5258 cpu_manager.go:414] "containerMap: removing container" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="kube-rbac-proxy-thanos" Nov 04 05:28:56.966991 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.966991 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="kube-rbac-proxy-thanos" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.966998 5258 cpu_manager.go:414] "containerMap: removing container" podUID="4a120e88-95d0-4176-ab05-586d4c2e7dab" containerName="insights-gathering" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967003 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a120e88-95d0-4176-ab05-586d4c2e7dab" containerName="insights-gathering" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967012 5258 cpu_manager.go:414] "containerMap: removing container" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="init-config-reloader" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967017 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="init-config-reloader" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967023 5258 cpu_manager.go:414] "containerMap: removing container" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="prom-label-proxy" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967028 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="prom-label-proxy" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967047 5258 cpu_manager.go:414] "containerMap: removing container" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="thanos-sidecar" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967052 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="thanos-sidecar" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967065 5258 cpu_manager.go:414] "containerMap: removing container" podUID="11b27b07-7ada-441c-8799-e3d3d111bc18" containerName="console" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967069 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11b27b07-7ada-441c-8799-e3d3d111bc18" containerName="console" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967074 5258 cpu_manager.go:414] "containerMap: removing container" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="kube-rbac-proxy-web" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967079 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="kube-rbac-proxy-web" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967085 5258 cpu_manager.go:414] "containerMap: removing container" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="kube-rbac-proxy-metric" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967089 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="kube-rbac-proxy-metric" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967095 5258 cpu_manager.go:414] "containerMap: removing container" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="kube-rbac-proxy-web" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967099 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="kube-rbac-proxy-web" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967106 5258 cpu_manager.go:414] "containerMap: removing container" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="alertmanager" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967110 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="alertmanager" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967120 5258 cpu_manager.go:414] "containerMap: removing container" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="kube-rbac-proxy" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967124 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="kube-rbac-proxy" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967131 5258 cpu_manager.go:414] "containerMap: removing container" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="init-config-reloader" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967135 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="init-config-reloader" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967141 5258 cpu_manager.go:414] "containerMap: removing container" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="prometheus" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967146 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="prometheus" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967188 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="kube-rbac-proxy" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967195 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="prometheus" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967202 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="config-reloader" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967208 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="config-reloader" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967213 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="kube-rbac-proxy" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967219 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="kube-rbac-proxy-thanos" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967224 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a120e88-95d0-4176-ab05-586d4c2e7dab" containerName="insights-gathering" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967229 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="prom-label-proxy" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967234 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="thanos-sidecar" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967241 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="11b27b07-7ada-441c-8799-e3d3d111bc18" containerName="console" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967246 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="kube-rbac-proxy-web" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967251 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="kube-rbac-proxy-metric" Nov 04 05:28:56.967238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967256 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" containerName="kube-rbac-proxy-web" Nov 04 05:28:56.968413 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.967262 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3ea1a41-8e39-4d16-ae87-0e030773dc09" containerName="alertmanager" Nov 04 05:28:56.970937 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:56.970920 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.022025 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.021954 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Nov 04 05:28:57.022150 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.022061 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Nov 04 05:28:57.022209 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.022156 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Nov 04 05:28:57.022209 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.022192 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-mjb6f\"" Nov 04 05:28:57.022291 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.022246 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Nov 04 05:28:57.022291 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.022254 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Nov 04 05:28:57.022388 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.022341 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Nov 04 05:28:57.022388 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.022372 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Nov 04 05:28:57.022489 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.022396 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Nov 04 05:28:57.022489 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.022394 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-e13puemqhc024\"" Nov 04 05:28:57.022489 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.022408 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Nov 04 05:28:57.022489 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.022427 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Nov 04 05:28:57.022489 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.022437 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Nov 04 05:28:57.022489 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.022456 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Nov 04 05:28:57.070050 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.070011 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Nov 04 05:28:57.070161 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.070146 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Nov 04 05:28:57.078813 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.078792 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 05:28:57.083930 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.083907 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.084060 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.083942 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/add07311-410f-451b-9010-f8f88a49d33a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.084060 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.083967 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.084149 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.084067 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-web-config\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.084149 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.084104 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/add07311-410f-451b-9010-f8f88a49d33a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.084228 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.084171 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/add07311-410f-451b-9010-f8f88a49d33a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.084276 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.084228 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.084330 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.084315 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-config\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.084371 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.084356 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/add07311-410f-451b-9010-f8f88a49d33a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.084443 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.084428 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/add07311-410f-451b-9010-f8f88a49d33a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.084499 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.084457 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.084538 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.084527 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.084587 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.084557 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/add07311-410f-451b-9010-f8f88a49d33a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.084623 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.084607 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/add07311-410f-451b-9010-f8f88a49d33a-config-out\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.084672 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.084636 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.084729 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.084711 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/add07311-410f-451b-9010-f8f88a49d33a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.084771 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.084738 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/add07311-410f-451b-9010-f8f88a49d33a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.084771 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.084754 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/add07311-410f-451b-9010-f8f88a49d33a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.084852 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.084834 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpz4d\" (UniqueName: \"kubernetes.io/projected/add07311-410f-451b-9010-f8f88a49d33a-kube-api-access-bpz4d\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.084912 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.084895 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.185999 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.185968 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/add07311-410f-451b-9010-f8f88a49d33a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.186193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.186005 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/add07311-410f-451b-9010-f8f88a49d33a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.186193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.186021 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.186193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.186056 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.186193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.186073 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/add07311-410f-451b-9010-f8f88a49d33a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.186193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.186091 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/add07311-410f-451b-9010-f8f88a49d33a-config-out\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.186193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.186109 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.186193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.186129 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/add07311-410f-451b-9010-f8f88a49d33a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.186193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.186150 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/add07311-410f-451b-9010-f8f88a49d33a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.186193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.186174 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/add07311-410f-451b-9010-f8f88a49d33a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.186573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.186213 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpz4d\" (UniqueName: \"kubernetes.io/projected/add07311-410f-451b-9010-f8f88a49d33a-kube-api-access-bpz4d\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.186573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.186244 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.186573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.186275 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.186573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.186298 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/add07311-410f-451b-9010-f8f88a49d33a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.186573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.186318 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.186573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.186343 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-web-config\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.186573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.186365 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/add07311-410f-451b-9010-f8f88a49d33a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.186573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.186394 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/add07311-410f-451b-9010-f8f88a49d33a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.186573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.186424 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.186573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.186490 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-config\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.187106 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.186996 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/add07311-410f-451b-9010-f8f88a49d33a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.187178 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.187148 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/add07311-410f-451b-9010-f8f88a49d33a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.187257 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.187230 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/add07311-410f-451b-9010-f8f88a49d33a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.188196 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.187876 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/add07311-410f-451b-9010-f8f88a49d33a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.189381 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.189075 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/add07311-410f-451b-9010-f8f88a49d33a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.189381 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.189172 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/add07311-410f-451b-9010-f8f88a49d33a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.189381 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.189345 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/add07311-410f-451b-9010-f8f88a49d33a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.189705 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.189681 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.190023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.190003 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/add07311-410f-451b-9010-f8f88a49d33a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.190194 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.190090 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.190305 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.190119 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-config\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.190305 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.190294 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.190678 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.190638 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.191559 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.191537 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.191656 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.191594 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-web-config\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.192259 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.192235 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/add07311-410f-451b-9010-f8f88a49d33a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.192360 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.192329 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.192360 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.192335 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/add07311-410f-451b-9010-f8f88a49d33a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.192491 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.192475 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/add07311-410f-451b-9010-f8f88a49d33a-config-out\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.248162 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.248127 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpz4d\" (UniqueName: \"kubernetes.io/projected/add07311-410f-451b-9010-f8f88a49d33a-kube-api-access-bpz4d\") pod \"prometheus-k8s-0\" (UID: \"add07311-410f-451b-9010-f8f88a49d33a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.303868 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.303793 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:28:57.725664 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.725637 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 05:28:57.728426 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:28:57.728398 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadd07311_410f_451b_9010_f8f88a49d33a.slice/crio-0781e769dbbf4a7a81de77169cc9398b1c82013bcc195d069044994af018235c WatchSource:0}: Error finding container 0781e769dbbf4a7a81de77169cc9398b1c82013bcc195d069044994af018235c: Status 404 returned error can't find the container with id 0781e769dbbf4a7a81de77169cc9398b1c82013bcc195d069044994af018235c Nov 04 05:28:57.758094 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:57.758059 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"add07311-410f-451b-9010-f8f88a49d33a","Type":"ContainerStarted","Data":"0781e769dbbf4a7a81de77169cc9398b1c82013bcc195d069044994af018235c"} Nov 04 05:28:58.267412 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:58.267382 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e42acc16-8c50-4990-8f7e-35ac3df287f2" path="/var/lib/kubelet/pods/e42acc16-8c50-4990-8f7e-35ac3df287f2/volumes" Nov 04 05:28:58.764083 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:58.764017 5258 generic.go:356] "Generic (PLEG): container finished" podID="add07311-410f-451b-9010-f8f88a49d33a" containerID="82f3c415e8daa792119553780787a33e954ae2f4b942d2a34579f41735c92699" exitCode=0 Nov 04 05:28:58.764276 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:58.764110 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"add07311-410f-451b-9010-f8f88a49d33a","Type":"ContainerDied","Data":"82f3c415e8daa792119553780787a33e954ae2f4b942d2a34579f41735c92699"} Nov 04 05:28:59.771532 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:59.771498 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"add07311-410f-451b-9010-f8f88a49d33a","Type":"ContainerStarted","Data":"f14beba2355f922130d33274eee819a17837e2e7a0b61c54e13ad7a7bc7d84a9"} Nov 04 05:28:59.771905 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:59.771588 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"add07311-410f-451b-9010-f8f88a49d33a","Type":"ContainerStarted","Data":"e00a3d8ff75b21773a3c8391eeb3a6f65bf8ea0961e3777eeead183e0acf03c2"} Nov 04 05:28:59.771905 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:59.771614 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"add07311-410f-451b-9010-f8f88a49d33a","Type":"ContainerStarted","Data":"4767a97f101267b4215a2f9c26e0267f2824d60741f99193ce5e2e0d8db47cbd"} Nov 04 05:28:59.771905 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:59.771637 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"add07311-410f-451b-9010-f8f88a49d33a","Type":"ContainerStarted","Data":"985e8cdfbca47d0859905d06de398918006d35798ea1f0a30dd8762cac070c90"} Nov 04 05:28:59.771905 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:59.771663 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"add07311-410f-451b-9010-f8f88a49d33a","Type":"ContainerStarted","Data":"6952c86b6a450c27e8174e8e8f76ae7b322d0822902d45523ae0310d38da5adf"} Nov 04 05:28:59.771905 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:59.771686 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"add07311-410f-451b-9010-f8f88a49d33a","Type":"ContainerStarted","Data":"9eab2b7d088744320ecf3d834aae26f068e1889da5c16542442d911fc6707c00"} Nov 04 05:28:59.900968 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:28:59.900917 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.884984533 podStartE2EDuration="3.900902957s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.918424ms" isStatefulPod=false podCreationTimestamp="2026-11-04 05:28:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:28:59.900023552 +0000 UTC m=+73.779416024" watchObservedRunningTime="2026-11-04 05:28:59.900902957 +0000 UTC m=+73.780295430" Nov 04 05:29:02.304381 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:29:02.304348 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:22.779974 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:29:22.779936 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:29:52.779324 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:29:52.779283 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:29:57.304386 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:29:57.304356 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:57.319635 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:29:57.319609 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:29:58.005895 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:29:58.005865 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 05:30:22.780525 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:30:22.780483 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:30:52.779326 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:30:52.779289 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:31:22.779802 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:31:22.779766 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:31:52.779499 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:31:52.779458 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:31:58.291232 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.291202 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2"] Nov 04 05:31:58.294692 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.294672 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" Nov 04 05:31:58.400914 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.400875 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2"] Nov 04 05:31:58.418276 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.418247 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zf2m\" (UniqueName: \"kubernetes.io/projected/6bad1c0a-1d05-4ab4-b341-8d33bbbe2112-kube-api-access-7zf2m\") pod \"opendatahub-operator-controller-manager-c98788444-lwsx2\" (UID: \"6bad1c0a-1d05-4ab4-b341-8d33bbbe2112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" Nov 04 05:31:58.418399 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.418285 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6bad1c0a-1d05-4ab4-b341-8d33bbbe2112-manifests\") pod \"opendatahub-operator-controller-manager-c98788444-lwsx2\" (UID: \"6bad1c0a-1d05-4ab4-b341-8d33bbbe2112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" Nov 04 05:31:58.418441 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.418388 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6bad1c0a-1d05-4ab4-b341-8d33bbbe2112-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c98788444-lwsx2\" (UID: \"6bad1c0a-1d05-4ab4-b341-8d33bbbe2112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" Nov 04 05:31:58.418480 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.418450 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6bad1c0a-1d05-4ab4-b341-8d33bbbe2112-metrics-cert\") pod \"opendatahub-operator-controller-manager-c98788444-lwsx2\" (UID: \"6bad1c0a-1d05-4ab4-b341-8d33bbbe2112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" Nov 04 05:31:58.418480 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.418474 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bad1c0a-1d05-4ab4-b341-8d33bbbe2112-webhook-cert\") pod \"opendatahub-operator-controller-manager-c98788444-lwsx2\" (UID: \"6bad1c0a-1d05-4ab4-b341-8d33bbbe2112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" Nov 04 05:31:58.418574 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.418553 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bad1c0a-1d05-4ab4-b341-8d33bbbe2112-tmp\") pod \"opendatahub-operator-controller-manager-c98788444-lwsx2\" (UID: \"6bad1c0a-1d05-4ab4-b341-8d33bbbe2112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" Nov 04 05:31:58.426745 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.426727 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\"" Nov 04 05:31:58.426866 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.426790 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\"" Nov 04 05:31:58.429679 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.429659 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\"" Nov 04 05:31:58.429679 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.429677 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\"" Nov 04 05:31:58.429788 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.429763 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\"" Nov 04 05:31:58.429788 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.429770 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-ssrcr\"" Nov 04 05:31:58.519998 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.519969 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6bad1c0a-1d05-4ab4-b341-8d33bbbe2112-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c98788444-lwsx2\" (UID: \"6bad1c0a-1d05-4ab4-b341-8d33bbbe2112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" Nov 04 05:31:58.520167 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.520011 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6bad1c0a-1d05-4ab4-b341-8d33bbbe2112-metrics-cert\") pod \"opendatahub-operator-controller-manager-c98788444-lwsx2\" (UID: \"6bad1c0a-1d05-4ab4-b341-8d33bbbe2112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" Nov 04 05:31:58.520167 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.520056 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bad1c0a-1d05-4ab4-b341-8d33bbbe2112-webhook-cert\") pod \"opendatahub-operator-controller-manager-c98788444-lwsx2\" (UID: \"6bad1c0a-1d05-4ab4-b341-8d33bbbe2112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" Nov 04 05:31:58.520284 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.520226 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bad1c0a-1d05-4ab4-b341-8d33bbbe2112-tmp\") pod \"opendatahub-operator-controller-manager-c98788444-lwsx2\" (UID: \"6bad1c0a-1d05-4ab4-b341-8d33bbbe2112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" Nov 04 05:31:58.520354 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.520333 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zf2m\" (UniqueName: \"kubernetes.io/projected/6bad1c0a-1d05-4ab4-b341-8d33bbbe2112-kube-api-access-7zf2m\") pod \"opendatahub-operator-controller-manager-c98788444-lwsx2\" (UID: \"6bad1c0a-1d05-4ab4-b341-8d33bbbe2112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" Nov 04 05:31:58.520410 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.520390 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6bad1c0a-1d05-4ab4-b341-8d33bbbe2112-manifests\") pod \"opendatahub-operator-controller-manager-c98788444-lwsx2\" (UID: \"6bad1c0a-1d05-4ab4-b341-8d33bbbe2112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" Nov 04 05:31:58.520797 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.520770 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6bad1c0a-1d05-4ab4-b341-8d33bbbe2112-manifests\") pod \"opendatahub-operator-controller-manager-c98788444-lwsx2\" (UID: \"6bad1c0a-1d05-4ab4-b341-8d33bbbe2112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" Nov 04 05:31:58.522615 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.522589 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6bad1c0a-1d05-4ab4-b341-8d33bbbe2112-webhook-cert\") pod \"opendatahub-operator-controller-manager-c98788444-lwsx2\" (UID: \"6bad1c0a-1d05-4ab4-b341-8d33bbbe2112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" Nov 04 05:31:58.522719 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.522619 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6bad1c0a-1d05-4ab4-b341-8d33bbbe2112-metrics-cert\") pod \"opendatahub-operator-controller-manager-c98788444-lwsx2\" (UID: \"6bad1c0a-1d05-4ab4-b341-8d33bbbe2112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" Nov 04 05:31:58.522760 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.522733 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6bad1c0a-1d05-4ab4-b341-8d33bbbe2112-tmp\") pod \"opendatahub-operator-controller-manager-c98788444-lwsx2\" (UID: \"6bad1c0a-1d05-4ab4-b341-8d33bbbe2112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" Nov 04 05:31:58.522894 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.522879 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6bad1c0a-1d05-4ab4-b341-8d33bbbe2112-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c98788444-lwsx2\" (UID: \"6bad1c0a-1d05-4ab4-b341-8d33bbbe2112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" Nov 04 05:31:58.583109 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.583022 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zf2m\" (UniqueName: \"kubernetes.io/projected/6bad1c0a-1d05-4ab4-b341-8d33bbbe2112-kube-api-access-7zf2m\") pod \"opendatahub-operator-controller-manager-c98788444-lwsx2\" (UID: \"6bad1c0a-1d05-4ab4-b341-8d33bbbe2112\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" Nov 04 05:31:58.609910 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:58.609883 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" Nov 04 05:31:59.113149 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:31:59.113119 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bad1c0a_1d05_4ab4_b341_8d33bbbe2112.slice/crio-979c2677f234a1d9cbf21dbdc0b9219b2cebc2817075f0dd16ec2aeca78c739a WatchSource:0}: Error finding container 979c2677f234a1d9cbf21dbdc0b9219b2cebc2817075f0dd16ec2aeca78c739a: Status 404 returned error can't find the container with id 979c2677f234a1d9cbf21dbdc0b9219b2cebc2817075f0dd16ec2aeca78c739a Nov 04 05:31:59.114415 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:59.114387 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2"] Nov 04 05:31:59.429506 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:31:59.429423 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" event={"ID":"6bad1c0a-1d05-4ab4-b341-8d33bbbe2112","Type":"ContainerStarted","Data":"979c2677f234a1d9cbf21dbdc0b9219b2cebc2817075f0dd16ec2aeca78c739a"} Nov 04 05:32:03.448601 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:03.448564 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" event={"ID":"6bad1c0a-1d05-4ab4-b341-8d33bbbe2112","Type":"ContainerStarted","Data":"e3347ff547cd710515b994d68fd53bbfa8633f5d611551caffabc35bb1585b37"} Nov 04 05:32:04.454075 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:04.454022 5258 generic.go:356] "Generic (PLEG): container finished" podID="6bad1c0a-1d05-4ab4-b341-8d33bbbe2112" containerID="e3347ff547cd710515b994d68fd53bbfa8633f5d611551caffabc35bb1585b37" exitCode=0 Nov 04 05:32:04.454493 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:04.454077 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" event={"ID":"6bad1c0a-1d05-4ab4-b341-8d33bbbe2112","Type":"ContainerDied","Data":"e3347ff547cd710515b994d68fd53bbfa8633f5d611551caffabc35bb1585b37"} Nov 04 05:32:06.466361 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:06.466324 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" event={"ID":"6bad1c0a-1d05-4ab4-b341-8d33bbbe2112","Type":"ContainerStarted","Data":"83eab6b6d39ea150b042f8427d0bcab84b12c0a4115348e2497a2f5f1e6c4ed8"} Nov 04 05:32:06.466812 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:06.466461 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" Nov 04 05:32:06.761428 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:06.761312 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" podStartSLOduration=3.350002418 podStartE2EDuration="8.761293915s" totalImagesPullingTime="4.878279662s" totalInitContainerRuntime="533.011835ms" isStatefulPod=false podCreationTimestamp="2026-11-04 05:31:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:32:06.708324579 +0000 UTC m=+260.587717084" watchObservedRunningTime="2026-11-04 05:32:06.761293915 +0000 UTC m=+260.640686389" Nov 04 05:32:17.473578 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:17.473542 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-c98788444-lwsx2" Nov 04 05:32:22.780299 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:32:22.780259 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:32:25.507478 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:25.507407 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-889s2"] Nov 04 05:32:25.509802 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:25.509781 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-889s2" Nov 04 05:32:25.542527 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:25.542499 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c13b4684-27ed-4378-aca4-c6606d36233f-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-889s2\" (UID: \"c13b4684-27ed-4378-aca4-c6606d36233f\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-889s2" Nov 04 05:32:25.542671 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:25.542555 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxw42\" (UniqueName: \"kubernetes.io/projected/c13b4684-27ed-4378-aca4-c6606d36233f-kube-api-access-bxw42\") pod \"cert-manager-webhook-5dbbbf75cc-889s2\" (UID: \"c13b4684-27ed-4378-aca4-c6606d36233f\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-889s2" Nov 04 05:32:25.567668 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:25.567641 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Nov 04 05:32:25.623020 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:25.622986 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Nov 04 05:32:25.623972 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:25.623950 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-889s2"] Nov 04 05:32:25.643537 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:25.643502 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxw42\" (UniqueName: \"kubernetes.io/projected/c13b4684-27ed-4378-aca4-c6606d36233f-kube-api-access-bxw42\") pod \"cert-manager-webhook-5dbbbf75cc-889s2\" (UID: \"c13b4684-27ed-4378-aca4-c6606d36233f\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-889s2" Nov 04 05:32:25.643687 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:25.643570 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c13b4684-27ed-4378-aca4-c6606d36233f-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-889s2\" (UID: \"c13b4684-27ed-4378-aca4-c6606d36233f\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-889s2" Nov 04 05:32:25.678434 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:25.678408 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-7rxm9\"" Nov 04 05:32:25.711604 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:25.711577 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c13b4684-27ed-4378-aca4-c6606d36233f-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-889s2\" (UID: \"c13b4684-27ed-4378-aca4-c6606d36233f\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-889s2" Nov 04 05:32:25.711737 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:25.711647 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxw42\" (UniqueName: \"kubernetes.io/projected/c13b4684-27ed-4378-aca4-c6606d36233f-kube-api-access-bxw42\") pod \"cert-manager-webhook-5dbbbf75cc-889s2\" (UID: \"c13b4684-27ed-4378-aca4-c6606d36233f\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-889s2" Nov 04 05:32:25.829626 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:25.829538 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-889s2" Nov 04 05:32:26.213246 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:26.213213 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-889s2"] Nov 04 05:32:26.221577 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:32:26.221547 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc13b4684_27ed_4378_aca4_c6606d36233f.slice/crio-cd5aab4b6b5baeaeb4a4a18a6424dae1495e9d36f688c327ffa1eb39575f1ae4 WatchSource:0}: Error finding container cd5aab4b6b5baeaeb4a4a18a6424dae1495e9d36f688c327ffa1eb39575f1ae4: Status 404 returned error can't find the container with id cd5aab4b6b5baeaeb4a4a18a6424dae1495e9d36f688c327ffa1eb39575f1ae4 Nov 04 05:32:26.543795 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:26.543712 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-889s2" event={"ID":"c13b4684-27ed-4378-aca4-c6606d36233f","Type":"ContainerStarted","Data":"cd5aab4b6b5baeaeb4a4a18a6424dae1495e9d36f688c327ffa1eb39575f1ae4"} Nov 04 05:32:27.679079 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:27.679043 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-5mj86"] Nov 04 05:32:27.682732 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:27.682708 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5mj86" Nov 04 05:32:27.742330 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:27.742292 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-dnpmf\"" Nov 04 05:32:27.763702 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:27.763666 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hm67\" (UniqueName: \"kubernetes.io/projected/88c5502c-3f50-423b-aaf4-82d64c95cd8f-kube-api-access-2hm67\") pod \"cert-manager-cainjector-5c57bbd4fc-5mj86\" (UID: \"88c5502c-3f50-423b-aaf4-82d64c95cd8f\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5mj86" Nov 04 05:32:27.763848 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:27.763718 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88c5502c-3f50-423b-aaf4-82d64c95cd8f-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-5mj86\" (UID: \"88c5502c-3f50-423b-aaf4-82d64c95cd8f\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5mj86" Nov 04 05:32:27.793341 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:27.793305 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-5mj86"] Nov 04 05:32:27.865211 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:27.865177 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hm67\" (UniqueName: \"kubernetes.io/projected/88c5502c-3f50-423b-aaf4-82d64c95cd8f-kube-api-access-2hm67\") pod \"cert-manager-cainjector-5c57bbd4fc-5mj86\" (UID: \"88c5502c-3f50-423b-aaf4-82d64c95cd8f\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5mj86" Nov 04 05:32:27.865376 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:27.865217 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88c5502c-3f50-423b-aaf4-82d64c95cd8f-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-5mj86\" (UID: \"88c5502c-3f50-423b-aaf4-82d64c95cd8f\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5mj86" Nov 04 05:32:27.927433 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:27.927401 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88c5502c-3f50-423b-aaf4-82d64c95cd8f-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-5mj86\" (UID: \"88c5502c-3f50-423b-aaf4-82d64c95cd8f\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5mj86" Nov 04 05:32:27.927555 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:27.927455 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hm67\" (UniqueName: \"kubernetes.io/projected/88c5502c-3f50-423b-aaf4-82d64c95cd8f-kube-api-access-2hm67\") pod \"cert-manager-cainjector-5c57bbd4fc-5mj86\" (UID: \"88c5502c-3f50-423b-aaf4-82d64c95cd8f\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5mj86" Nov 04 05:32:27.993774 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:27.993691 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5mj86" Nov 04 05:32:28.401367 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:28.401330 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-5mj86"] Nov 04 05:32:28.409507 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:32:28.409476 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88c5502c_3f50_423b_aaf4_82d64c95cd8f.slice/crio-6518dbaa97d67ebd0987211d85e854a477f9a6d3a6085635a2f946856349a53a WatchSource:0}: Error finding container 6518dbaa97d67ebd0987211d85e854a477f9a6d3a6085635a2f946856349a53a: Status 404 returned error can't find the container with id 6518dbaa97d67ebd0987211d85e854a477f9a6d3a6085635a2f946856349a53a Nov 04 05:32:28.554885 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:28.554815 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5mj86" event={"ID":"88c5502c-3f50-423b-aaf4-82d64c95cd8f","Type":"ContainerStarted","Data":"6518dbaa97d67ebd0987211d85e854a477f9a6d3a6085635a2f946856349a53a"} Nov 04 05:32:32.575057 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:32.574998 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-889s2" event={"ID":"c13b4684-27ed-4378-aca4-c6606d36233f","Type":"ContainerStarted","Data":"94457ec3c4e3b9882f65d98f1a59707357cc4f520036ddc2285922f9d8347bb6"} Nov 04 05:32:32.575466 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:32.575096 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-889s2" Nov 04 05:32:32.729833 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:32.729777 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-889s2" podStartSLOduration=2.241186801 podStartE2EDuration="7.729765045s" totalImagesPullingTime="5.488578244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:32:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:32:32.728175371 +0000 UTC m=+286.607567854" watchObservedRunningTime="2026-11-04 05:32:32.729765045 +0000 UTC m=+286.609157517" Nov 04 05:32:33.585945 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:33.585903 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5mj86" event={"ID":"88c5502c-3f50-423b-aaf4-82d64c95cd8f","Type":"ContainerStarted","Data":"5e700dfb4839341466695d304f62fac44a2d956cc12c78aad7544d3238dd6a8f"} Nov 04 05:32:33.704550 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:33.704496 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-5mj86" podStartSLOduration=2.522528324 podStartE2EDuration="6.704482278s" totalImagesPullingTime="4.181953954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:32:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:32:33.703749867 +0000 UTC m=+287.583142344" watchObservedRunningTime="2026-11-04 05:32:33.704482278 +0000 UTC m=+287.583874744" Nov 04 05:32:34.154953 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:34.154914 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5c6f4696b9-vlb8m"] Nov 04 05:32:34.158401 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:34.158377 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-vlb8m" Nov 04 05:32:34.214666 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:34.214639 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Nov 04 05:32:34.214666 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:34.214653 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Nov 04 05:32:34.214864 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:34.214672 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Nov 04 05:32:34.214864 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:34.214664 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-x2vv4\"" Nov 04 05:32:34.214864 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:34.214727 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Nov 04 05:32:34.222091 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:34.222073 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcb78eed-38d1-4fa3-9697-99f6b0883659-tmp\") pod \"kube-auth-proxy-5c6f4696b9-vlb8m\" (UID: \"fcb78eed-38d1-4fa3-9697-99f6b0883659\") " pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-vlb8m" Nov 04 05:32:34.222191 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:34.222121 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcb78eed-38d1-4fa3-9697-99f6b0883659-tls-certs\") pod \"kube-auth-proxy-5c6f4696b9-vlb8m\" (UID: \"fcb78eed-38d1-4fa3-9697-99f6b0883659\") " pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-vlb8m" Nov 04 05:32:34.222191 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:34.222147 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdrml\" (UniqueName: \"kubernetes.io/projected/fcb78eed-38d1-4fa3-9697-99f6b0883659-kube-api-access-wdrml\") pod \"kube-auth-proxy-5c6f4696b9-vlb8m\" (UID: \"fcb78eed-38d1-4fa3-9697-99f6b0883659\") " pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-vlb8m" Nov 04 05:32:34.222918 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:34.222898 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c6f4696b9-vlb8m"] Nov 04 05:32:34.323240 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:34.323198 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcb78eed-38d1-4fa3-9697-99f6b0883659-tmp\") pod \"kube-auth-proxy-5c6f4696b9-vlb8m\" (UID: \"fcb78eed-38d1-4fa3-9697-99f6b0883659\") " pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-vlb8m" Nov 04 05:32:34.323441 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:34.323263 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcb78eed-38d1-4fa3-9697-99f6b0883659-tls-certs\") pod \"kube-auth-proxy-5c6f4696b9-vlb8m\" (UID: \"fcb78eed-38d1-4fa3-9697-99f6b0883659\") " pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-vlb8m" Nov 04 05:32:34.323441 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:34.323293 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdrml\" (UniqueName: \"kubernetes.io/projected/fcb78eed-38d1-4fa3-9697-99f6b0883659-kube-api-access-wdrml\") pod \"kube-auth-proxy-5c6f4696b9-vlb8m\" (UID: \"fcb78eed-38d1-4fa3-9697-99f6b0883659\") " pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-vlb8m" Nov 04 05:32:34.325617 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:34.325588 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fcb78eed-38d1-4fa3-9697-99f6b0883659-tmp\") pod \"kube-auth-proxy-5c6f4696b9-vlb8m\" (UID: \"fcb78eed-38d1-4fa3-9697-99f6b0883659\") " pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-vlb8m" Nov 04 05:32:34.325849 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:34.325825 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcb78eed-38d1-4fa3-9697-99f6b0883659-tls-certs\") pod \"kube-auth-proxy-5c6f4696b9-vlb8m\" (UID: \"fcb78eed-38d1-4fa3-9697-99f6b0883659\") " pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-vlb8m" Nov 04 05:32:34.386068 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:34.386012 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdrml\" (UniqueName: \"kubernetes.io/projected/fcb78eed-38d1-4fa3-9697-99f6b0883659-kube-api-access-wdrml\") pod \"kube-auth-proxy-5c6f4696b9-vlb8m\" (UID: \"fcb78eed-38d1-4fa3-9697-99f6b0883659\") " pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-vlb8m" Nov 04 05:32:34.470816 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:34.470739 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-vlb8m" Nov 04 05:32:34.866071 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:34.866025 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c6f4696b9-vlb8m"] Nov 04 05:32:34.874577 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:32:34.874547 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcb78eed_38d1_4fa3_9697_99f6b0883659.slice/crio-76d8cab90b2d4673a211d67838e622f6edcfdbde7ecdf864700f580fd49aebaf WatchSource:0}: Error finding container 76d8cab90b2d4673a211d67838e622f6edcfdbde7ecdf864700f580fd49aebaf: Status 404 returned error can't find the container with id 76d8cab90b2d4673a211d67838e622f6edcfdbde7ecdf864700f580fd49aebaf Nov 04 05:32:35.595496 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:35.595459 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-vlb8m" event={"ID":"fcb78eed-38d1-4fa3-9697-99f6b0883659","Type":"ContainerStarted","Data":"76d8cab90b2d4673a211d67838e622f6edcfdbde7ecdf864700f580fd49aebaf"} Nov 04 05:32:37.282196 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:37.282162 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-sbvs6"] Nov 04 05:32:37.285450 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:37.285428 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-sbvs6" Nov 04 05:32:37.352422 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:37.352390 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-przf2\" (UniqueName: \"kubernetes.io/projected/c465d7f4-a204-40f0-b99a-dd60576b1709-kube-api-access-przf2\") pod \"cert-manager-597b95cb84-sbvs6\" (UID: \"c465d7f4-a204-40f0-b99a-dd60576b1709\") " pod="cert-manager/cert-manager-597b95cb84-sbvs6" Nov 04 05:32:37.352609 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:37.352453 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c465d7f4-a204-40f0-b99a-dd60576b1709-bound-sa-token\") pod \"cert-manager-597b95cb84-sbvs6\" (UID: \"c465d7f4-a204-40f0-b99a-dd60576b1709\") " pod="cert-manager/cert-manager-597b95cb84-sbvs6" Nov 04 05:32:37.392490 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:37.392456 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-fbnj9\"" Nov 04 05:32:37.393694 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:37.393666 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-sbvs6"] Nov 04 05:32:37.454018 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:37.453985 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c465d7f4-a204-40f0-b99a-dd60576b1709-bound-sa-token\") pod \"cert-manager-597b95cb84-sbvs6\" (UID: \"c465d7f4-a204-40f0-b99a-dd60576b1709\") " pod="cert-manager/cert-manager-597b95cb84-sbvs6" Nov 04 05:32:37.454199 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:37.454161 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-przf2\" (UniqueName: \"kubernetes.io/projected/c465d7f4-a204-40f0-b99a-dd60576b1709-kube-api-access-przf2\") pod \"cert-manager-597b95cb84-sbvs6\" (UID: \"c465d7f4-a204-40f0-b99a-dd60576b1709\") " pod="cert-manager/cert-manager-597b95cb84-sbvs6" Nov 04 05:32:37.522260 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:37.522229 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c465d7f4-a204-40f0-b99a-dd60576b1709-bound-sa-token\") pod \"cert-manager-597b95cb84-sbvs6\" (UID: \"c465d7f4-a204-40f0-b99a-dd60576b1709\") " pod="cert-manager/cert-manager-597b95cb84-sbvs6" Nov 04 05:32:37.522461 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:37.522438 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-przf2\" (UniqueName: \"kubernetes.io/projected/c465d7f4-a204-40f0-b99a-dd60576b1709-kube-api-access-przf2\") pod \"cert-manager-597b95cb84-sbvs6\" (UID: \"c465d7f4-a204-40f0-b99a-dd60576b1709\") " pod="cert-manager/cert-manager-597b95cb84-sbvs6" Nov 04 05:32:37.598002 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:37.597923 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-sbvs6" Nov 04 05:32:37.978855 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:37.978821 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-sbvs6"] Nov 04 05:32:37.987272 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:32:37.987235 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc465d7f4_a204_40f0_b99a_dd60576b1709.slice/crio-d3dbcf872f61808c9a616c64bb0d0721f61675c74bb24e7028c62c2a1d8314dc WatchSource:0}: Error finding container d3dbcf872f61808c9a616c64bb0d0721f61675c74bb24e7028c62c2a1d8314dc: Status 404 returned error can't find the container with id d3dbcf872f61808c9a616c64bb0d0721f61675c74bb24e7028c62c2a1d8314dc Nov 04 05:32:38.588972 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:38.588942 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-889s2" Nov 04 05:32:38.610123 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:38.610086 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-sbvs6" event={"ID":"c465d7f4-a204-40f0-b99a-dd60576b1709","Type":"ContainerStarted","Data":"d2d97be0454af96bb0f6e8bf756ca0cab3047e571ba705a595a3647862b9a0dc"} Nov 04 05:32:38.610123 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:38.610129 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-sbvs6" event={"ID":"c465d7f4-a204-40f0-b99a-dd60576b1709","Type":"ContainerStarted","Data":"d3dbcf872f61808c9a616c64bb0d0721f61675c74bb24e7028c62c2a1d8314dc"} Nov 04 05:32:38.839632 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:38.839538 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-sbvs6" podStartSLOduration=1.8395228000000001 podStartE2EDuration="1.8395228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:32:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:32:38.838572893 +0000 UTC m=+292.717965366" watchObservedRunningTime="2026-11-04 05:32:38.8395228 +0000 UTC m=+292.718915272" Nov 04 05:32:40.620490 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:40.620451 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-vlb8m" event={"ID":"fcb78eed-38d1-4fa3-9697-99f6b0883659","Type":"ContainerStarted","Data":"ee64f3d074d1a861e2015ff3f9961427b15c3c3c050ee2a5da362239eb5af0ec"} Nov 04 05:32:40.752536 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:40.752442 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5c6f4696b9-vlb8m" podStartSLOduration=1.250478658 podStartE2EDuration="6.752426511s" totalImagesPullingTime="5.501947853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:32:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:32:40.752097102 +0000 UTC m=+294.631489575" watchObservedRunningTime="2026-11-04 05:32:40.752426511 +0000 UTC m=+294.631818983" Nov 04 05:32:41.476363 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:41.476333 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-nw6wr"] Nov 04 05:32:41.479888 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:41.479865 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-nw6wr" Nov 04 05:32:41.534714 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:41.534682 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Nov 04 05:32:41.534871 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:41.534768 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Nov 04 05:32:41.534871 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:41.534771 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Nov 04 05:32:41.534871 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:41.534803 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6rl8q\"" Nov 04 05:32:41.588558 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:41.588527 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-nw6wr"] Nov 04 05:32:41.588925 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:41.588908 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqvdr\" (UniqueName: \"kubernetes.io/projected/987e3827-629f-42e6-bfbe-f0645cf3608a-kube-api-access-pqvdr\") pod \"limitador-limitador-74dfcfb894-nw6wr\" (UID: \"987e3827-629f-42e6-bfbe-f0645cf3608a\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-nw6wr" Nov 04 05:32:41.589009 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:41.588945 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/987e3827-629f-42e6-bfbe-f0645cf3608a-config-file\") pod \"limitador-limitador-74dfcfb894-nw6wr\" (UID: \"987e3827-629f-42e6-bfbe-f0645cf3608a\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-nw6wr" Nov 04 05:32:41.690468 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:41.690431 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqvdr\" (UniqueName: \"kubernetes.io/projected/987e3827-629f-42e6-bfbe-f0645cf3608a-kube-api-access-pqvdr\") pod \"limitador-limitador-74dfcfb894-nw6wr\" (UID: \"987e3827-629f-42e6-bfbe-f0645cf3608a\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-nw6wr" Nov 04 05:32:41.690960 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:41.690484 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/987e3827-629f-42e6-bfbe-f0645cf3608a-config-file\") pod \"limitador-limitador-74dfcfb894-nw6wr\" (UID: \"987e3827-629f-42e6-bfbe-f0645cf3608a\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-nw6wr" Nov 04 05:32:41.691208 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:41.691187 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/987e3827-629f-42e6-bfbe-f0645cf3608a-config-file\") pod \"limitador-limitador-74dfcfb894-nw6wr\" (UID: \"987e3827-629f-42e6-bfbe-f0645cf3608a\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-nw6wr" Nov 04 05:32:41.717453 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:41.717417 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-nw6wr"] Nov 04 05:32:41.754177 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:41.754114 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqvdr\" (UniqueName: \"kubernetes.io/projected/987e3827-629f-42e6-bfbe-f0645cf3608a-kube-api-access-pqvdr\") pod \"limitador-limitador-74dfcfb894-nw6wr\" (UID: \"987e3827-629f-42e6-bfbe-f0645cf3608a\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-nw6wr" Nov 04 05:32:41.792742 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:41.792711 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-nw6wr" Nov 04 05:32:42.171651 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:42.171614 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-nw6wr"] Nov 04 05:32:42.181593 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:32:42.181534 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod987e3827_629f_42e6_bfbe_f0645cf3608a.slice/crio-bae0fb47beae48cd39c3945c3baa43ba161ad62122b8e10b5ce6b54010030fc8 WatchSource:0}: Error finding container bae0fb47beae48cd39c3945c3baa43ba161ad62122b8e10b5ce6b54010030fc8: Status 404 returned error can't find the container with id bae0fb47beae48cd39c3945c3baa43ba161ad62122b8e10b5ce6b54010030fc8 Nov 04 05:32:42.629085 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:42.629051 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-nw6wr" event={"ID":"987e3827-629f-42e6-bfbe-f0645cf3608a","Type":"ContainerStarted","Data":"bae0fb47beae48cd39c3945c3baa43ba161ad62122b8e10b5ce6b54010030fc8"} Nov 04 05:32:45.986101 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:45.986060 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:32:45.997262 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:45.997005 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-6jlll" Nov 04 05:32:46.101306 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:46.101221 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:32:46.132475 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:46.132439 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxql2\" (UniqueName: \"kubernetes.io/projected/6a427024-ba5b-4a22-80da-2cb2f8b5bd13-kube-api-access-dxql2\") pod \"limitador-limitador-7c9b87d574-6jlll\" (UID: \"6a427024-ba5b-4a22-80da-2cb2f8b5bd13\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-6jlll" Nov 04 05:32:46.132627 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:46.132567 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6a427024-ba5b-4a22-80da-2cb2f8b5bd13-config-file\") pod \"limitador-limitador-7c9b87d574-6jlll\" (UID: \"6a427024-ba5b-4a22-80da-2cb2f8b5bd13\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-6jlll" Nov 04 05:32:46.233640 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:46.233605 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxql2\" (UniqueName: \"kubernetes.io/projected/6a427024-ba5b-4a22-80da-2cb2f8b5bd13-kube-api-access-dxql2\") pod \"limitador-limitador-7c9b87d574-6jlll\" (UID: \"6a427024-ba5b-4a22-80da-2cb2f8b5bd13\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-6jlll" Nov 04 05:32:46.233792 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:46.233682 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6a427024-ba5b-4a22-80da-2cb2f8b5bd13-config-file\") pod \"limitador-limitador-7c9b87d574-6jlll\" (UID: \"6a427024-ba5b-4a22-80da-2cb2f8b5bd13\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-6jlll" Nov 04 05:32:46.234285 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:46.234266 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6a427024-ba5b-4a22-80da-2cb2f8b5bd13-config-file\") pod \"limitador-limitador-7c9b87d574-6jlll\" (UID: \"6a427024-ba5b-4a22-80da-2cb2f8b5bd13\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-6jlll" Nov 04 05:32:46.300100 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:46.300072 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxql2\" (UniqueName: \"kubernetes.io/projected/6a427024-ba5b-4a22-80da-2cb2f8b5bd13-kube-api-access-dxql2\") pod \"limitador-limitador-7c9b87d574-6jlll\" (UID: \"6a427024-ba5b-4a22-80da-2cb2f8b5bd13\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-6jlll" Nov 04 05:32:46.309766 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:46.309740 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:32:46.312022 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:46.312001 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-6jlll" Nov 04 05:32:46.646718 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:46.646686 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-nw6wr" event={"ID":"987e3827-629f-42e6-bfbe-f0645cf3608a","Type":"ContainerStarted","Data":"3f38371ee0d4fdd633553752de39f00213c514e03ebebf8f6a083caedbc3a8be"} Nov 04 05:32:46.646864 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:46.646780 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-nw6wr" Nov 04 05:32:46.710476 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:46.710446 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:32:46.714205 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:32:46.714173 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a427024_ba5b_4a22_80da_2cb2f8b5bd13.slice/crio-1e2dfc775705bbe88ea883eb33d66307c662c56b6fed4b3aebbc858dddaf3a68 WatchSource:0}: Error finding container 1e2dfc775705bbe88ea883eb33d66307c662c56b6fed4b3aebbc858dddaf3a68: Status 404 returned error can't find the container with id 1e2dfc775705bbe88ea883eb33d66307c662c56b6fed4b3aebbc858dddaf3a68 Nov 04 05:32:46.785734 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:46.785689 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-nw6wr" podStartSLOduration=1.7005518130000001 podStartE2EDuration="5.785675219s" totalImagesPullingTime="4.085123406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:32:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:32:46.78489276 +0000 UTC m=+300.664285232" watchObservedRunningTime="2026-11-04 05:32:46.785675219 +0000 UTC m=+300.665067692" Nov 04 05:32:47.652479 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:47.652442 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-6jlll" event={"ID":"6a427024-ba5b-4a22-80da-2cb2f8b5bd13","Type":"ContainerStarted","Data":"cf6057e7ed87f111386e2e82f621aef56e0a9fda5bd71345fe2355990c55be5b"} Nov 04 05:32:47.652979 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:47.652531 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-6jlll" event={"ID":"6a427024-ba5b-4a22-80da-2cb2f8b5bd13","Type":"ContainerStarted","Data":"1e2dfc775705bbe88ea883eb33d66307c662c56b6fed4b3aebbc858dddaf3a68"} Nov 04 05:32:47.775637 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:47.775587 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-6jlll" podStartSLOduration=2.775573558 podStartE2EDuration="2.775573558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:32:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:32:47.774763997 +0000 UTC m=+301.654156469" watchObservedRunningTime="2026-11-04 05:32:47.775573558 +0000 UTC m=+301.654966009" Nov 04 05:32:48.656520 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:48.656490 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-6jlll" Nov 04 05:32:52.780790 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:32:52.780752 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:32:57.653290 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:57.653263 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-nw6wr" Nov 04 05:32:59.662434 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:59.662399 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-6jlll" Nov 04 05:32:59.850242 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:59.850206 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-nw6wr"] Nov 04 05:32:59.850418 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:32:59.850398 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-nw6wr" podUID="987e3827-629f-42e6-bfbe-f0645cf3608a" containerName="limitador" containerID="cri-o://3f38371ee0d4fdd633553752de39f00213c514e03ebebf8f6a083caedbc3a8be" gracePeriod=30 Nov 04 05:33:00.389012 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.388990 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-nw6wr" Nov 04 05:33:00.458484 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.458415 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/987e3827-629f-42e6-bfbe-f0645cf3608a-config-file\" (UniqueName: \"kubernetes.io/configmap/987e3827-629f-42e6-bfbe-f0645cf3608a-config-file\") pod \"987e3827-629f-42e6-bfbe-f0645cf3608a\" (UID: \"987e3827-629f-42e6-bfbe-f0645cf3608a\") " Nov 04 05:33:00.458608 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.458531 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/987e3827-629f-42e6-bfbe-f0645cf3608a-kube-api-access-pqvdr\" (UniqueName: \"kubernetes.io/projected/987e3827-629f-42e6-bfbe-f0645cf3608a-kube-api-access-pqvdr\") pod \"987e3827-629f-42e6-bfbe-f0645cf3608a\" (UID: \"987e3827-629f-42e6-bfbe-f0645cf3608a\") " Nov 04 05:33:00.458772 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.458750 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/987e3827-629f-42e6-bfbe-f0645cf3608a-config-file" pod "987e3827-629f-42e6-bfbe-f0645cf3608a" (UID: "987e3827-629f-42e6-bfbe-f0645cf3608a"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:33:00.460649 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.460629 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/987e3827-629f-42e6-bfbe-f0645cf3608a-kube-api-access-pqvdr" pod "987e3827-629f-42e6-bfbe-f0645cf3608a" (UID: "987e3827-629f-42e6-bfbe-f0645cf3608a"). InnerVolumeSpecName "kube-api-access-pqvdr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:33:00.559385 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.559351 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqvdr\" (UniqueName: \"kubernetes.io/projected/987e3827-629f-42e6-bfbe-f0645cf3608a-kube-api-access-pqvdr\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:33:00.559385 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.559380 5258 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/987e3827-629f-42e6-bfbe-f0645cf3608a-config-file\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:33:00.707500 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.707465 5258 generic.go:356] "Generic (PLEG): container finished" podID="987e3827-629f-42e6-bfbe-f0645cf3608a" containerID="3f38371ee0d4fdd633553752de39f00213c514e03ebebf8f6a083caedbc3a8be" exitCode=0 Nov 04 05:33:00.707931 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.707513 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-nw6wr" Nov 04 05:33:00.707931 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.707552 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-nw6wr" event={"ID":"987e3827-629f-42e6-bfbe-f0645cf3608a","Type":"ContainerDied","Data":"3f38371ee0d4fdd633553752de39f00213c514e03ebebf8f6a083caedbc3a8be"} Nov 04 05:33:00.707931 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.707610 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-nw6wr" event={"ID":"987e3827-629f-42e6-bfbe-f0645cf3608a","Type":"ContainerDied","Data":"bae0fb47beae48cd39c3945c3baa43ba161ad62122b8e10b5ce6b54010030fc8"} Nov 04 05:33:00.707931 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.707638 5258 scope.go:122] "RemoveContainer" containerID="3f38371ee0d4fdd633553752de39f00213c514e03ebebf8f6a083caedbc3a8be" Nov 04 05:33:00.715891 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.715872 5258 scope.go:122] "RemoveContainer" containerID="3f38371ee0d4fdd633553752de39f00213c514e03ebebf8f6a083caedbc3a8be" Nov 04 05:33:00.716191 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:33:00.716172 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f38371ee0d4fdd633553752de39f00213c514e03ebebf8f6a083caedbc3a8be\": container with ID starting with 3f38371ee0d4fdd633553752de39f00213c514e03ebebf8f6a083caedbc3a8be not found: ID does not exist" containerID="3f38371ee0d4fdd633553752de39f00213c514e03ebebf8f6a083caedbc3a8be" Nov 04 05:33:00.716259 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.716200 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f38371ee0d4fdd633553752de39f00213c514e03ebebf8f6a083caedbc3a8be"} err="failed to get container status \"3f38371ee0d4fdd633553752de39f00213c514e03ebebf8f6a083caedbc3a8be\": rpc error: code = NotFound desc = could not find container \"3f38371ee0d4fdd633553752de39f00213c514e03ebebf8f6a083caedbc3a8be\": container with ID starting with 3f38371ee0d4fdd633553752de39f00213c514e03ebebf8f6a083caedbc3a8be not found: ID does not exist" Nov 04 05:33:00.837021 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.836989 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-c7d795d5b-jzx2v"] Nov 04 05:33:00.837363 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.837350 5258 cpu_manager.go:414] "containerMap: removing container" podUID="987e3827-629f-42e6-bfbe-f0645cf3608a" containerName="limitador" Nov 04 05:33:00.837400 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.837365 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="987e3827-629f-42e6-bfbe-f0645cf3608a" containerName="limitador" Nov 04 05:33:00.837444 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.837431 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="987e3827-629f-42e6-bfbe-f0645cf3608a" containerName="limitador" Nov 04 05:33:00.841716 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.841698 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-c7d795d5b-jzx2v" Nov 04 05:33:00.862550 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.862528 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d-manager-config\") pod \"jobset-controller-manager-c7d795d5b-jzx2v\" (UID: \"7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d\") " pod="openshift-jobset-operator/jobset-controller-manager-c7d795d5b-jzx2v" Nov 04 05:33:00.862657 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.862564 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d-metrics-certs\") pod \"jobset-controller-manager-c7d795d5b-jzx2v\" (UID: \"7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d\") " pod="openshift-jobset-operator/jobset-controller-manager-c7d795d5b-jzx2v" Nov 04 05:33:00.862657 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.862590 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25q9v\" (UniqueName: \"kubernetes.io/projected/7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d-kube-api-access-25q9v\") pod \"jobset-controller-manager-c7d795d5b-jzx2v\" (UID: \"7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d\") " pod="openshift-jobset-operator/jobset-controller-manager-c7d795d5b-jzx2v" Nov 04 05:33:00.862745 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.862700 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d-cert\") pod \"jobset-controller-manager-c7d795d5b-jzx2v\" (UID: \"7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d\") " pod="openshift-jobset-operator/jobset-controller-manager-c7d795d5b-jzx2v" Nov 04 05:33:00.888706 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.888680 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-nw6wr"] Nov 04 05:33:00.892503 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.892482 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-nw6wr"] Nov 04 05:33:00.943362 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.943322 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\"" Nov 04 05:33:00.943362 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.943352 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\"" Nov 04 05:33:00.943568 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.943338 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\"" Nov 04 05:33:00.943568 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.943338 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-7bjhh\"" Nov 04 05:33:00.943568 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.943386 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\"" Nov 04 05:33:00.944903 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.944883 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-c7d795d5b-jzx2v"] Nov 04 05:33:00.963141 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.963081 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d-metrics-certs\") pod \"jobset-controller-manager-c7d795d5b-jzx2v\" (UID: \"7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d\") " pod="openshift-jobset-operator/jobset-controller-manager-c7d795d5b-jzx2v" Nov 04 05:33:00.963141 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.963114 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25q9v\" (UniqueName: \"kubernetes.io/projected/7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d-kube-api-access-25q9v\") pod \"jobset-controller-manager-c7d795d5b-jzx2v\" (UID: \"7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d\") " pod="openshift-jobset-operator/jobset-controller-manager-c7d795d5b-jzx2v" Nov 04 05:33:00.963141 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.963137 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d-cert\") pod \"jobset-controller-manager-c7d795d5b-jzx2v\" (UID: \"7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d\") " pod="openshift-jobset-operator/jobset-controller-manager-c7d795d5b-jzx2v" Nov 04 05:33:00.963312 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.963199 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d-manager-config\") pod \"jobset-controller-manager-c7d795d5b-jzx2v\" (UID: \"7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d\") " pod="openshift-jobset-operator/jobset-controller-manager-c7d795d5b-jzx2v" Nov 04 05:33:00.963897 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.963873 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d-manager-config\") pod \"jobset-controller-manager-c7d795d5b-jzx2v\" (UID: \"7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d\") " pod="openshift-jobset-operator/jobset-controller-manager-c7d795d5b-jzx2v" Nov 04 05:33:00.965550 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.965524 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d-cert\") pod \"jobset-controller-manager-c7d795d5b-jzx2v\" (UID: \"7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d\") " pod="openshift-jobset-operator/jobset-controller-manager-c7d795d5b-jzx2v" Nov 04 05:33:00.965637 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:00.965564 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d-metrics-certs\") pod \"jobset-controller-manager-c7d795d5b-jzx2v\" (UID: \"7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d\") " pod="openshift-jobset-operator/jobset-controller-manager-c7d795d5b-jzx2v" Nov 04 05:33:01.000909 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:01.000884 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\"" Nov 04 05:33:01.034010 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:01.033971 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25q9v\" (UniqueName: \"kubernetes.io/projected/7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d-kube-api-access-25q9v\") pod \"jobset-controller-manager-c7d795d5b-jzx2v\" (UID: \"7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d\") " pod="openshift-jobset-operator/jobset-controller-manager-c7d795d5b-jzx2v" Nov 04 05:33:01.153733 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:01.153691 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-c7d795d5b-jzx2v" Nov 04 05:33:01.538141 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:01.538061 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-c7d795d5b-jzx2v"] Nov 04 05:33:01.542707 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:33:01.542681 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c9dda9a_e6d0_4a2c_aa37_6aee16cb9c2d.slice/crio-7a7ebc69eb1ed5df8454b15543c0dcff865d7a3244c8e5cdbcbae5610e0dd917 WatchSource:0}: Error finding container 7a7ebc69eb1ed5df8454b15543c0dcff865d7a3244c8e5cdbcbae5610e0dd917: Status 404 returned error can't find the container with id 7a7ebc69eb1ed5df8454b15543c0dcff865d7a3244c8e5cdbcbae5610e0dd917 Nov 04 05:33:01.544757 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:01.544742 5258 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 05:33:01.714726 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:01.714684 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-c7d795d5b-jzx2v" event={"ID":"7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d","Type":"ContainerStarted","Data":"7a7ebc69eb1ed5df8454b15543c0dcff865d7a3244c8e5cdbcbae5610e0dd917"} Nov 04 05:33:02.269232 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:02.269202 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="987e3827-629f-42e6-bfbe-f0645cf3608a" path="/var/lib/kubelet/pods/987e3827-629f-42e6-bfbe-f0645cf3608a/volumes" Nov 04 05:33:07.743378 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:07.743340 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-c7d795d5b-jzx2v" event={"ID":"7c9dda9a-e6d0-4a2c-aa37-6aee16cb9c2d","Type":"ContainerStarted","Data":"88feed8c5b164e02f48014028ce3aff104dea07f942813569bab9bb712ea5e89"} Nov 04 05:33:07.743901 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:07.743467 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-c7d795d5b-jzx2v" Nov 04 05:33:07.872697 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:07.872647 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-c7d795d5b-jzx2v" podStartSLOduration=2.529760817 podStartE2EDuration="7.872631966s" totalImagesPullingTime="5.342871149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:33:07.871454365 +0000 UTC m=+321.750846837" watchObservedRunningTime="2026-11-04 05:33:07.872631966 +0000 UTC m=+321.752024439" Nov 04 05:33:18.752793 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:33:18.752761 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-c7d795d5b-jzx2v" Nov 04 05:33:22.781706 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:33:22.781669 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:33:52.780768 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:33:52.780734 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:34:14.375807 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:34:14.375731 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:34:22.780335 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:34:22.780298 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:34:52.780731 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:34:52.780692 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:35:22.779366 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:35:22.779326 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:35:52.779441 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:35:52.779398 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:36:05.898423 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:05.898384 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv"] Nov 04 05:36:05.901979 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:05.901956 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:05.977758 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:05.977720 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzk68\" (UniqueName: \"kubernetes.io/projected/580ba144-f615-4558-ad64-17f0839f01ab-kube-api-access-qzk68\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv\" (UID: \"580ba144-f615-4558-ad64-17f0839f01ab\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:05.977931 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:05.977824 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/580ba144-f615-4558-ad64-17f0839f01ab-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv\" (UID: \"580ba144-f615-4558-ad64-17f0839f01ab\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:05.977931 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:05.977874 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/580ba144-f615-4558-ad64-17f0839f01ab-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv\" (UID: \"580ba144-f615-4558-ad64-17f0839f01ab\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:05.977931 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:05.977920 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/580ba144-f615-4558-ad64-17f0839f01ab-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv\" (UID: \"580ba144-f615-4558-ad64-17f0839f01ab\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:05.978087 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:05.978073 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/580ba144-f615-4558-ad64-17f0839f01ab-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv\" (UID: \"580ba144-f615-4558-ad64-17f0839f01ab\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:05.978135 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:05.978111 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/580ba144-f615-4558-ad64-17f0839f01ab-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv\" (UID: \"580ba144-f615-4558-ad64-17f0839f01ab\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:06.000312 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:06.000278 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-528s5\"" Nov 04 05:36:06.000312 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:06.000296 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Nov 04 05:36:06.000487 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:06.000357 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Nov 04 05:36:06.000487 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:06.000369 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Nov 04 05:36:06.001626 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:06.001606 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv"] Nov 04 05:36:06.079052 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:06.079002 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/580ba144-f615-4558-ad64-17f0839f01ab-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv\" (UID: \"580ba144-f615-4558-ad64-17f0839f01ab\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:06.079227 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:06.079067 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/580ba144-f615-4558-ad64-17f0839f01ab-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv\" (UID: \"580ba144-f615-4558-ad64-17f0839f01ab\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:06.079227 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:06.079116 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/580ba144-f615-4558-ad64-17f0839f01ab-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv\" (UID: \"580ba144-f615-4558-ad64-17f0839f01ab\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:06.079227 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:06.079168 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/580ba144-f615-4558-ad64-17f0839f01ab-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv\" (UID: \"580ba144-f615-4558-ad64-17f0839f01ab\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:06.079227 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:06.079207 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/580ba144-f615-4558-ad64-17f0839f01ab-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv\" (UID: \"580ba144-f615-4558-ad64-17f0839f01ab\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:06.079399 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:06.079372 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzk68\" (UniqueName: \"kubernetes.io/projected/580ba144-f615-4558-ad64-17f0839f01ab-kube-api-access-qzk68\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv\" (UID: \"580ba144-f615-4558-ad64-17f0839f01ab\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:06.079556 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:06.079532 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/580ba144-f615-4558-ad64-17f0839f01ab-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv\" (UID: \"580ba144-f615-4558-ad64-17f0839f01ab\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:06.079556 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:06.079547 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/580ba144-f615-4558-ad64-17f0839f01ab-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv\" (UID: \"580ba144-f615-4558-ad64-17f0839f01ab\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:06.079681 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:06.079593 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/580ba144-f615-4558-ad64-17f0839f01ab-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv\" (UID: \"580ba144-f615-4558-ad64-17f0839f01ab\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:06.081213 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:06.081188 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/580ba144-f615-4558-ad64-17f0839f01ab-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv\" (UID: \"580ba144-f615-4558-ad64-17f0839f01ab\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:06.081633 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:06.081611 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/580ba144-f615-4558-ad64-17f0839f01ab-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv\" (UID: \"580ba144-f615-4558-ad64-17f0839f01ab\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:06.139522 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:06.139487 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzk68\" (UniqueName: \"kubernetes.io/projected/580ba144-f615-4558-ad64-17f0839f01ab-kube-api-access-qzk68\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv\" (UID: \"580ba144-f615-4558-ad64-17f0839f01ab\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:06.213206 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:06.213124 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:06.259908 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:06.259875 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:36:06.597930 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:06.597857 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv"] Nov 04 05:36:06.606565 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:36:06.606537 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod580ba144_f615_4558_ad64_17f0839f01ab.slice/crio-49019eecddef4eef6a286939708003c9836be0e89b128bc0e535d4d3b46bfd47 WatchSource:0}: Error finding container 49019eecddef4eef6a286939708003c9836be0e89b128bc0e535d4d3b46bfd47: Status 404 returned error can't find the container with id 49019eecddef4eef6a286939708003c9836be0e89b128bc0e535d4d3b46bfd47 Nov 04 05:36:07.498463 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:07.498421 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" event={"ID":"580ba144-f615-4558-ad64-17f0839f01ab","Type":"ContainerStarted","Data":"49019eecddef4eef6a286939708003c9836be0e89b128bc0e535d4d3b46bfd47"} Nov 04 05:36:08.504374 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:08.504285 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" event={"ID":"580ba144-f615-4558-ad64-17f0839f01ab","Type":"ContainerStarted","Data":"95b6522be1b708ab95b6eddd1eba3e54be2bf8f5d28c00ae6a77367d8b82d8d1"} Nov 04 05:36:08.616488 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:08.616440 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" podStartSLOduration=2.088935144 podStartE2EDuration="3.616426349s" totalImagesPullingTime="1.527491205s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:36:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:36:08.615352171 +0000 UTC m=+502.494744646" watchObservedRunningTime="2026-11-04 05:36:08.616426349 +0000 UTC m=+502.495818818" Nov 04 05:36:09.213283 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:09.213244 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:16.213594 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:16.213553 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:16.225856 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:16.225831 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:16.547855 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:16.547771 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv" Nov 04 05:36:19.630762 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:19.630724 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:36:22.779923 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:36:22.779887 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:36:27.307719 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.307684 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd"] Nov 04 05:36:27.311415 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.311399 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:27.372865 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.372832 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q7r9\" (UniqueName: \"kubernetes.io/projected/538f3de1-190f-4977-98e7-aef0bfcaa4b8-kube-api-access-4q7r9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd\" (UID: \"538f3de1-190f-4977-98e7-aef0bfcaa4b8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:27.373054 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.372886 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/538f3de1-190f-4977-98e7-aef0bfcaa4b8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd\" (UID: \"538f3de1-190f-4977-98e7-aef0bfcaa4b8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:27.373109 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.373053 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/538f3de1-190f-4977-98e7-aef0bfcaa4b8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd\" (UID: \"538f3de1-190f-4977-98e7-aef0bfcaa4b8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:27.373109 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.373094 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/538f3de1-190f-4977-98e7-aef0bfcaa4b8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd\" (UID: \"538f3de1-190f-4977-98e7-aef0bfcaa4b8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:27.373235 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.373134 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/538f3de1-190f-4977-98e7-aef0bfcaa4b8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd\" (UID: \"538f3de1-190f-4977-98e7-aef0bfcaa4b8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:27.373235 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.373162 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/538f3de1-190f-4977-98e7-aef0bfcaa4b8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd\" (UID: \"538f3de1-190f-4977-98e7-aef0bfcaa4b8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:27.419118 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.419087 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Nov 04 05:36:27.420387 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.420365 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd"] Nov 04 05:36:27.473697 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.473665 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/538f3de1-190f-4977-98e7-aef0bfcaa4b8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd\" (UID: \"538f3de1-190f-4977-98e7-aef0bfcaa4b8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:27.473887 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.473708 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/538f3de1-190f-4977-98e7-aef0bfcaa4b8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd\" (UID: \"538f3de1-190f-4977-98e7-aef0bfcaa4b8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:27.473887 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.473759 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q7r9\" (UniqueName: \"kubernetes.io/projected/538f3de1-190f-4977-98e7-aef0bfcaa4b8-kube-api-access-4q7r9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd\" (UID: \"538f3de1-190f-4977-98e7-aef0bfcaa4b8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:27.473887 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.473786 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/538f3de1-190f-4977-98e7-aef0bfcaa4b8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd\" (UID: \"538f3de1-190f-4977-98e7-aef0bfcaa4b8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:27.473887 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.473850 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/538f3de1-190f-4977-98e7-aef0bfcaa4b8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd\" (UID: \"538f3de1-190f-4977-98e7-aef0bfcaa4b8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:27.473887 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.473879 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/538f3de1-190f-4977-98e7-aef0bfcaa4b8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd\" (UID: \"538f3de1-190f-4977-98e7-aef0bfcaa4b8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:27.474166 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.474146 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/538f3de1-190f-4977-98e7-aef0bfcaa4b8-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd\" (UID: \"538f3de1-190f-4977-98e7-aef0bfcaa4b8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:27.474228 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.474204 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/538f3de1-190f-4977-98e7-aef0bfcaa4b8-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd\" (UID: \"538f3de1-190f-4977-98e7-aef0bfcaa4b8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:27.474265 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.474228 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/538f3de1-190f-4977-98e7-aef0bfcaa4b8-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd\" (UID: \"538f3de1-190f-4977-98e7-aef0bfcaa4b8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:27.476007 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.475975 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/538f3de1-190f-4977-98e7-aef0bfcaa4b8-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd\" (UID: \"538f3de1-190f-4977-98e7-aef0bfcaa4b8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:27.476298 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.476279 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/538f3de1-190f-4977-98e7-aef0bfcaa4b8-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd\" (UID: \"538f3de1-190f-4977-98e7-aef0bfcaa4b8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:27.536568 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.536540 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q7r9\" (UniqueName: \"kubernetes.io/projected/538f3de1-190f-4977-98e7-aef0bfcaa4b8-kube-api-access-4q7r9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd\" (UID: \"538f3de1-190f-4977-98e7-aef0bfcaa4b8\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:27.624522 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:27.624489 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:28.008331 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:28.008247 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd"] Nov 04 05:36:28.011854 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:36:28.011826 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod538f3de1_190f_4977_98e7_aef0bfcaa4b8.slice/crio-d3415799e6b4143cdba4e7eb1f7a1a8227198244f35e7b7a6ff4e6600f7a1e33 WatchSource:0}: Error finding container d3415799e6b4143cdba4e7eb1f7a1a8227198244f35e7b7a6ff4e6600f7a1e33: Status 404 returned error can't find the container with id d3415799e6b4143cdba4e7eb1f7a1a8227198244f35e7b7a6ff4e6600f7a1e33 Nov 04 05:36:28.590869 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:28.590834 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" event={"ID":"538f3de1-190f-4977-98e7-aef0bfcaa4b8","Type":"ContainerStarted","Data":"d3415799e6b4143cdba4e7eb1f7a1a8227198244f35e7b7a6ff4e6600f7a1e33"} Nov 04 05:36:29.599183 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:29.599151 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" event={"ID":"538f3de1-190f-4977-98e7-aef0bfcaa4b8","Type":"ContainerStarted","Data":"5bfe6747db6bff60198270f0a7e3cc5d6acf5c7e37578a0b2973928a3b1d71b0"} Nov 04 05:36:29.625185 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:29.625155 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:29.720694 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:29.720643 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" podStartSLOduration=2.124182975 podStartE2EDuration="2.720628867s" totalImagesPullingTime="596.445892ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:36:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:36:29.720557448 +0000 UTC m=+523.599949917" watchObservedRunningTime="2026-11-04 05:36:29.720628867 +0000 UTC m=+523.600021331" Nov 04 05:36:30.527116 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:30.527080 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:36:32.823821 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:32.823786 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n"] Nov 04 05:36:32.827029 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:32.827012 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:32.881236 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:32.881202 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Nov 04 05:36:32.926932 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:32.926899 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:32.926932 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:32.926936 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb6kk\" (UniqueName: \"kubernetes.io/projected/4a15bdbb-3628-4150-92f8-8d04c5c554b4-kube-api-access-sb6kk\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:32.927166 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:32.926966 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:32.927166 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:32.927081 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:32.927263 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:32.927174 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:32.927263 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:32.927216 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:32.927263 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:32.927238 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:32.927410 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:32.927288 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:32.927410 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:32.927376 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:32.932446 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:32.932426 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-ztc4z\"" Nov 04 05:36:32.933814 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:32.933790 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n"] Nov 04 05:36:33.028343 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:33.028315 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:33.028343 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:33.028348 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:33.028610 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:33.028376 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:33.028610 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:33.028442 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:33.028610 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:33.028475 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb6kk\" (UniqueName: \"kubernetes.io/projected/4a15bdbb-3628-4150-92f8-8d04c5c554b4-kube-api-access-sb6kk\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:33.028610 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:33.028524 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:33.028610 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:33.028547 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:33.028610 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:33.028594 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:33.028924 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:33.028637 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:33.028924 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:33.028789 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:33.029065 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:33.028922 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:33.029065 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:33.028976 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:33.029065 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:33.029019 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:33.029242 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:33.029220 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:33.030771 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:33.030750 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:33.030955 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:33.030938 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:33.091020 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:33.090943 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:33.091020 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:33.091001 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb6kk\" (UniqueName: \"kubernetes.io/projected/4a15bdbb-3628-4150-92f8-8d04c5c554b4-kube-api-access-sb6kk\") pod \"maas-default-gateway-openshift-default-778b7b5c47-vmk4n\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:33.138765 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:33.138728 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:33.581784 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:33.581750 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n"] Nov 04 05:36:33.585671 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:36:33.585640 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a15bdbb_3628_4150_92f8_8d04c5c554b4.slice/crio-e1b809a27d9f740437c7515980471b9a74367266e81021e92bc11e69a0a6fe29 WatchSource:0}: Error finding container e1b809a27d9f740437c7515980471b9a74367266e81021e92bc11e69a0a6fe29: Status 404 returned error can't find the container with id e1b809a27d9f740437c7515980471b9a74367266e81021e92bc11e69a0a6fe29 Nov 04 05:36:33.619024 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:33.618995 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" event={"ID":"4a15bdbb-3628-4150-92f8-8d04c5c554b4","Type":"ContainerStarted","Data":"e1b809a27d9f740437c7515980471b9a74367266e81021e92bc11e69a0a6fe29"} Nov 04 05:36:34.324135 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.324097 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx"] Nov 04 05:36:34.327607 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.327581 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:34.433010 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.432978 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Nov 04 05:36:34.434303 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.434281 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx"] Nov 04 05:36:34.442374 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.442349 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6b54269-2caf-4d78-82a1-5860b08bc41a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx\" (UID: \"e6b54269-2caf-4d78-82a1-5860b08bc41a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:34.442449 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.442377 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6b54269-2caf-4d78-82a1-5860b08bc41a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx\" (UID: \"e6b54269-2caf-4d78-82a1-5860b08bc41a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:34.442449 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.442404 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6b54269-2caf-4d78-82a1-5860b08bc41a-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx\" (UID: \"e6b54269-2caf-4d78-82a1-5860b08bc41a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:34.442547 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.442529 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6b54269-2caf-4d78-82a1-5860b08bc41a-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx\" (UID: \"e6b54269-2caf-4d78-82a1-5860b08bc41a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:34.442593 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.442575 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdnkz\" (UniqueName: \"kubernetes.io/projected/e6b54269-2caf-4d78-82a1-5860b08bc41a-kube-api-access-zdnkz\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx\" (UID: \"e6b54269-2caf-4d78-82a1-5860b08bc41a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:34.442713 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.442695 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6b54269-2caf-4d78-82a1-5860b08bc41a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx\" (UID: \"e6b54269-2caf-4d78-82a1-5860b08bc41a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:34.543396 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.543358 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6b54269-2caf-4d78-82a1-5860b08bc41a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx\" (UID: \"e6b54269-2caf-4d78-82a1-5860b08bc41a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:34.543584 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.543411 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6b54269-2caf-4d78-82a1-5860b08bc41a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx\" (UID: \"e6b54269-2caf-4d78-82a1-5860b08bc41a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:34.543584 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.543432 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6b54269-2caf-4d78-82a1-5860b08bc41a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx\" (UID: \"e6b54269-2caf-4d78-82a1-5860b08bc41a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:34.543584 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.543468 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6b54269-2caf-4d78-82a1-5860b08bc41a-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx\" (UID: \"e6b54269-2caf-4d78-82a1-5860b08bc41a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:34.543584 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.543499 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6b54269-2caf-4d78-82a1-5860b08bc41a-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx\" (UID: \"e6b54269-2caf-4d78-82a1-5860b08bc41a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:34.543584 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.543544 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdnkz\" (UniqueName: \"kubernetes.io/projected/e6b54269-2caf-4d78-82a1-5860b08bc41a-kube-api-access-zdnkz\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx\" (UID: \"e6b54269-2caf-4d78-82a1-5860b08bc41a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:34.543834 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.543806 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6b54269-2caf-4d78-82a1-5860b08bc41a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx\" (UID: \"e6b54269-2caf-4d78-82a1-5860b08bc41a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:34.543834 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.543821 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6b54269-2caf-4d78-82a1-5860b08bc41a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx\" (UID: \"e6b54269-2caf-4d78-82a1-5860b08bc41a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:34.543935 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.543882 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6b54269-2caf-4d78-82a1-5860b08bc41a-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx\" (UID: \"e6b54269-2caf-4d78-82a1-5860b08bc41a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:34.545776 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.545746 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6b54269-2caf-4d78-82a1-5860b08bc41a-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx\" (UID: \"e6b54269-2caf-4d78-82a1-5860b08bc41a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:34.546021 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.546003 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6b54269-2caf-4d78-82a1-5860b08bc41a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx\" (UID: \"e6b54269-2caf-4d78-82a1-5860b08bc41a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:34.606977 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.606942 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdnkz\" (UniqueName: \"kubernetes.io/projected/e6b54269-2caf-4d78-82a1-5860b08bc41a-kube-api-access-zdnkz\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx\" (UID: \"e6b54269-2caf-4d78-82a1-5860b08bc41a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:34.639597 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:34.639567 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:35.023318 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:35.023232 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx"] Nov 04 05:36:35.027354 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:36:35.027328 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b54269_2caf_4d78_82a1_5860b08bc41a.slice/crio-bc5e99930430b97d95552fd2c84e246156780d1b8b9ff0eb1dbbdb5be2187c0d WatchSource:0}: Error finding container bc5e99930430b97d95552fd2c84e246156780d1b8b9ff0eb1dbbdb5be2187c0d: Status 404 returned error can't find the container with id bc5e99930430b97d95552fd2c84e246156780d1b8b9ff0eb1dbbdb5be2187c0d Nov 04 05:36:35.629384 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:35.629359 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" event={"ID":"e6b54269-2caf-4d78-82a1-5860b08bc41a","Type":"ContainerStarted","Data":"bc5e99930430b97d95552fd2c84e246156780d1b8b9ff0eb1dbbdb5be2187c0d"} Nov 04 05:36:36.636643 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:36.636602 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" event={"ID":"e6b54269-2caf-4d78-82a1-5860b08bc41a","Type":"ContainerStarted","Data":"d27fd84614d2c58a98757218b2e00d8eadf20e87106e681f2b6f17b6527ca801"} Nov 04 05:36:36.640063 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:36.640028 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:36.807931 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:36.807868 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" podStartSLOduration=2.212061115 podStartE2EDuration="2.807851461s" totalImagesPullingTime="595.790346ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:36:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:36:36.806698045 +0000 UTC m=+530.686090521" watchObservedRunningTime="2026-11-04 05:36:36.807851461 +0000 UTC m=+530.687243935" Nov 04 05:36:37.625003 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:37.624970 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:37.644655 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:37.644620 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:37.662840 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:37.662813 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd" Nov 04 05:36:38.315544 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:38.315516 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:36:38.315633 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:38.315584 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:36:38.315633 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:38.315614 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:36:38.647132 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:38.647094 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" event={"ID":"4a15bdbb-3628-4150-92f8-8d04c5c554b4","Type":"ContainerStarted","Data":"5505c01d78631ee83602dc6d88fb84e53eca9d3e04bb2ccca1352b9e96f02c72"} Nov 04 05:36:38.771788 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:38.771716 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" podStartSLOduration=2.044692911 podStartE2EDuration="6.77169685s" totalImagesPullingTime="4.727003939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:36:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:36:38.768536524 +0000 UTC m=+532.647928998" watchObservedRunningTime="2026-11-04 05:36:38.77169685 +0000 UTC m=+532.651089324" Nov 04 05:36:39.153059 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:39.147774 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:39.153059 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:39.147919 5258 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-vmk4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Nov 04 05:36:39.153059 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:39.147966 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" podUID="4a15bdbb-3628-4150-92f8-8d04c5c554b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Nov 04 05:36:40.150070 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:40.146482 5258 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-vmk4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Nov 04 05:36:40.150070 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:40.146549 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" podUID="4a15bdbb-3628-4150-92f8-8d04c5c554b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Nov 04 05:36:41.146972 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:41.146490 5258 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-vmk4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Nov 04 05:36:41.146972 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:41.146559 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" podUID="4a15bdbb-3628-4150-92f8-8d04c5c554b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Nov 04 05:36:42.151060 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:42.146496 5258 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-vmk4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Nov 04 05:36:42.151060 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:42.146567 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" podUID="4a15bdbb-3628-4150-92f8-8d04c5c554b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Nov 04 05:36:43.146923 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:43.146464 5258 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-vmk4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Nov 04 05:36:43.146923 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:43.146529 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" podUID="4a15bdbb-3628-4150-92f8-8d04c5c554b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Nov 04 05:36:44.149058 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:44.148440 5258 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-vmk4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Nov 04 05:36:44.149058 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:44.148523 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" podUID="4a15bdbb-3628-4150-92f8-8d04c5c554b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Nov 04 05:36:44.640213 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:44.640168 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:44.653484 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:44.653447 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:44.692290 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:44.692260 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx" Nov 04 05:36:45.139483 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:45.139438 5258 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-vmk4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Nov 04 05:36:45.139669 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:45.139523 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" podUID="4a15bdbb-3628-4150-92f8-8d04c5c554b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Nov 04 05:36:46.305653 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:46.305622 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:36:46.911119 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:46.911089 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:46.911420 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:46.911391 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:46.912273 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:46.912253 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:36:47.956675 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:47.956641 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4"] Nov 04 05:36:47.959749 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:47.959733 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.095972 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.095933 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf896a57-35de-4283-a01d-ebb28df05d38-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.096215 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.095994 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnvkd\" (UniqueName: \"kubernetes.io/projected/bf896a57-35de-4283-a01d-ebb28df05d38-kube-api-access-xnvkd\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.096215 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.096086 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf896a57-35de-4283-a01d-ebb28df05d38-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.096215 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.096150 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf896a57-35de-4283-a01d-ebb28df05d38-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.096215 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.096176 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf896a57-35de-4283-a01d-ebb28df05d38-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.096440 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.096328 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf896a57-35de-4283-a01d-ebb28df05d38-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.096440 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.096378 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf896a57-35de-4283-a01d-ebb28df05d38-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.096516 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.096446 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf896a57-35de-4283-a01d-ebb28df05d38-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.096516 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.096473 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf896a57-35de-4283-a01d-ebb28df05d38-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.188076 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.188016 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4"] Nov 04 05:36:48.197774 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.197746 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf896a57-35de-4283-a01d-ebb28df05d38-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.197918 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.197782 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf896a57-35de-4283-a01d-ebb28df05d38-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.197918 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.197810 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf896a57-35de-4283-a01d-ebb28df05d38-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.197918 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.197826 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf896a57-35de-4283-a01d-ebb28df05d38-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.197918 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.197870 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf896a57-35de-4283-a01d-ebb28df05d38-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.197918 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.197895 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnvkd\" (UniqueName: \"kubernetes.io/projected/bf896a57-35de-4283-a01d-ebb28df05d38-kube-api-access-xnvkd\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.197918 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.197913 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf896a57-35de-4283-a01d-ebb28df05d38-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.198227 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.197941 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf896a57-35de-4283-a01d-ebb28df05d38-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.198227 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.197976 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf896a57-35de-4283-a01d-ebb28df05d38-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.198330 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.198255 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf896a57-35de-4283-a01d-ebb28df05d38-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.198330 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.198290 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf896a57-35de-4283-a01d-ebb28df05d38-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.198430 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.198362 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf896a57-35de-4283-a01d-ebb28df05d38-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.198480 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.198437 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf896a57-35de-4283-a01d-ebb28df05d38-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.198677 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.198656 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf896a57-35de-4283-a01d-ebb28df05d38-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.200214 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.200190 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf896a57-35de-4283-a01d-ebb28df05d38-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.200678 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.200652 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf896a57-35de-4283-a01d-ebb28df05d38-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.256804 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.256738 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf896a57-35de-4283-a01d-ebb28df05d38-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.257072 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.257050 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnvkd\" (UniqueName: \"kubernetes.io/projected/bf896a57-35de-4283-a01d-ebb28df05d38-kube-api-access-xnvkd\") pod \"maas-default-gateway-openshift-default-778b7b5c47-8spv4\" (UID: \"bf896a57-35de-4283-a01d-ebb28df05d38\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.271697 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.271672 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:48.710673 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.710640 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4"] Nov 04 05:36:48.718535 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:36:48.718507 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf896a57_35de_4283_a01d_ebb28df05d38.slice/crio-9dc9b768230ba64c8a058494d89db122c59b17a2f5ed5639c920502f42ca4390 WatchSource:0}: Error finding container 9dc9b768230ba64c8a058494d89db122c59b17a2f5ed5639c920502f42ca4390: Status 404 returned error can't find the container with id 9dc9b768230ba64c8a058494d89db122c59b17a2f5ed5639c920502f42ca4390 Nov 04 05:36:48.720740 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.720708 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:36:48.720800 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.720776 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:36:48.720840 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:48.720805 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:36:49.706054 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:49.705993 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" event={"ID":"bf896a57-35de-4283-a01d-ebb28df05d38","Type":"ContainerStarted","Data":"c9fac8637d526c488c0134bc3cd6c31bdf35357a22a859ef8f97eabf60157b42"} Nov 04 05:36:49.706544 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:49.706066 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" event={"ID":"bf896a57-35de-4283-a01d-ebb28df05d38","Type":"ContainerStarted","Data":"9dc9b768230ba64c8a058494d89db122c59b17a2f5ed5639c920502f42ca4390"} Nov 04 05:36:49.824613 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:49.824565 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" podStartSLOduration=2.824551101 podStartE2EDuration="2.824551101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:36:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:36:49.82359573 +0000 UTC m=+543.702988202" watchObservedRunningTime="2026-11-04 05:36:49.824551101 +0000 UTC m=+543.703943574" Nov 04 05:36:50.272781 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:50.272748 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:50.274087 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:50.274063 5258 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-8spv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Nov 04 05:36:50.274204 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:50.274109 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" podUID="bf896a57-35de-4283-a01d-ebb28df05d38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Nov 04 05:36:51.272683 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:51.272644 5258 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-8spv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Nov 04 05:36:51.273130 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:51.272702 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" podUID="bf896a57-35de-4283-a01d-ebb28df05d38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Nov 04 05:36:52.272214 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:52.272180 5258 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-8spv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Nov 04 05:36:52.272383 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:52.272232 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" podUID="bf896a57-35de-4283-a01d-ebb28df05d38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Nov 04 05:36:52.782454 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:36:52.782355 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:36:53.272824 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:53.272788 5258 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-8spv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Nov 04 05:36:53.272998 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:53.272837 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" podUID="bf896a57-35de-4283-a01d-ebb28df05d38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Nov 04 05:36:54.272481 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:54.272404 5258 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-8spv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Nov 04 05:36:54.272481 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:54.272452 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" podUID="bf896a57-35de-4283-a01d-ebb28df05d38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Nov 04 05:36:55.272117 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:55.272080 5258 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-8spv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Nov 04 05:36:55.272329 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:55.272142 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" podUID="bf896a57-35de-4283-a01d-ebb28df05d38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Nov 04 05:36:56.275425 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:56.275381 5258 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-8spv4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Nov 04 05:36:56.275837 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:56.275452 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" podUID="bf896a57-35de-4283-a01d-ebb28df05d38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Nov 04 05:36:57.275659 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:57.275626 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:57.276150 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:57.275926 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:36:57.276554 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:36:57.276537 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-8spv4" Nov 04 05:37:15.413561 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:15.413526 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-4rp6q"] Nov 04 05:37:15.419652 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:15.419631 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bb9489676-4rp6q" Nov 04 05:37:15.488110 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:15.488074 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/022ed9ce-af89-4b64-84cd-db4462575f88-plugins-config-volume\") pod \"payload-processing-7bb9489676-4rp6q\" (UID: \"022ed9ce-af89-4b64-84cd-db4462575f88\") " pod="openshift-ingress/payload-processing-7bb9489676-4rp6q" Nov 04 05:37:15.488591 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:15.488308 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ht8q\" (UniqueName: \"kubernetes.io/projected/022ed9ce-af89-4b64-84cd-db4462575f88-kube-api-access-9ht8q\") pod \"payload-processing-7bb9489676-4rp6q\" (UID: \"022ed9ce-af89-4b64-84cd-db4462575f88\") " pod="openshift-ingress/payload-processing-7bb9489676-4rp6q" Nov 04 05:37:15.525250 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:15.525210 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Nov 04 05:37:15.525441 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:15.525328 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-9xr6h\"" Nov 04 05:37:15.527733 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:15.527693 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-4rp6q"] Nov 04 05:37:15.589309 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:15.589264 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/022ed9ce-af89-4b64-84cd-db4462575f88-plugins-config-volume\") pod \"payload-processing-7bb9489676-4rp6q\" (UID: \"022ed9ce-af89-4b64-84cd-db4462575f88\") " pod="openshift-ingress/payload-processing-7bb9489676-4rp6q" Nov 04 05:37:15.589701 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:15.589676 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ht8q\" (UniqueName: \"kubernetes.io/projected/022ed9ce-af89-4b64-84cd-db4462575f88-kube-api-access-9ht8q\") pod \"payload-processing-7bb9489676-4rp6q\" (UID: \"022ed9ce-af89-4b64-84cd-db4462575f88\") " pod="openshift-ingress/payload-processing-7bb9489676-4rp6q" Nov 04 05:37:15.590189 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:15.590160 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/022ed9ce-af89-4b64-84cd-db4462575f88-plugins-config-volume\") pod \"payload-processing-7bb9489676-4rp6q\" (UID: \"022ed9ce-af89-4b64-84cd-db4462575f88\") " pod="openshift-ingress/payload-processing-7bb9489676-4rp6q" Nov 04 05:37:15.652530 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:15.652491 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ht8q\" (UniqueName: \"kubernetes.io/projected/022ed9ce-af89-4b64-84cd-db4462575f88-kube-api-access-9ht8q\") pod \"payload-processing-7bb9489676-4rp6q\" (UID: \"022ed9ce-af89-4b64-84cd-db4462575f88\") " pod="openshift-ingress/payload-processing-7bb9489676-4rp6q" Nov 04 05:37:15.732704 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:15.732616 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bb9489676-4rp6q" Nov 04 05:37:16.131238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:16.131200 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-4rp6q"] Nov 04 05:37:16.135388 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:37:16.135358 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod022ed9ce_af89_4b64_84cd_db4462575f88.slice/crio-1200b62946278db48cce83efe2200b07d726318e2f21b71f58883f118f94a695 WatchSource:0}: Error finding container 1200b62946278db48cce83efe2200b07d726318e2f21b71f58883f118f94a695: Status 404 returned error can't find the container with id 1200b62946278db48cce83efe2200b07d726318e2f21b71f58883f118f94a695 Nov 04 05:37:16.829716 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:16.829683 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bb9489676-4rp6q" event={"ID":"022ed9ce-af89-4b64-84cd-db4462575f88","Type":"ContainerStarted","Data":"1200b62946278db48cce83efe2200b07d726318e2f21b71f58883f118f94a695"} Nov 04 05:37:17.953201 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:17.953157 5258 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-vmk4n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 05:37:17.953685 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:17.953217 5258 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" podUID="4a15bdbb-3628-4150-92f8-8d04c5c554b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 05:37:19.848306 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:19.848272 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bb9489676-4rp6q" event={"ID":"022ed9ce-af89-4b64-84cd-db4462575f88","Type":"ContainerStarted","Data":"42aae88eb577f09feacb902898472c863e91f170fc56b19e7c01e7be76129f63"} Nov 04 05:37:19.848686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:19.848415 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bb9489676-4rp6q" Nov 04 05:37:19.968301 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:19.968251 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7bb9489676-4rp6q" podStartSLOduration=1.837383022 podStartE2EDuration="4.968237303s" totalImagesPullingTime="3.130854281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:37:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:37:19.966682435 +0000 UTC m=+573.846074908" watchObservedRunningTime="2026-11-04 05:37:19.968237303 +0000 UTC m=+573.847629776" Nov 04 05:37:22.779453 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:37:22.779417 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:37:24.229629 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:24.229595 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-57c96dfd77-nsdrn"] Nov 04 05:37:24.232295 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:24.232278 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-57c96dfd77-nsdrn" Nov 04 05:37:24.291363 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:24.291339 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\"" Nov 04 05:37:24.342825 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:24.342788 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-gzmwp\"" Nov 04 05:37:24.342999 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:24.342833 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Nov 04 05:37:24.342999 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:24.342902 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Nov 04 05:37:24.342999 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:24.342972 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\"" Nov 04 05:37:24.344331 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:24.344312 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-57c96dfd77-nsdrn"] Nov 04 05:37:24.376486 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:24.376453 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n49s5\" (UniqueName: \"kubernetes.io/projected/6acc41b2-6de5-4075-bbc2-78975d0b1f6a-kube-api-access-n49s5\") pod \"ai-gateway-controller-57c96dfd77-nsdrn\" (UID: \"6acc41b2-6de5-4075-bbc2-78975d0b1f6a\") " pod="opendatahub/ai-gateway-controller-57c96dfd77-nsdrn" Nov 04 05:37:24.376609 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:24.376587 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6acc41b2-6de5-4075-bbc2-78975d0b1f6a-metrics-certs\") pod \"ai-gateway-controller-57c96dfd77-nsdrn\" (UID: \"6acc41b2-6de5-4075-bbc2-78975d0b1f6a\") " pod="opendatahub/ai-gateway-controller-57c96dfd77-nsdrn" Nov 04 05:37:24.477906 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:24.477874 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6acc41b2-6de5-4075-bbc2-78975d0b1f6a-metrics-certs\") pod \"ai-gateway-controller-57c96dfd77-nsdrn\" (UID: \"6acc41b2-6de5-4075-bbc2-78975d0b1f6a\") " pod="opendatahub/ai-gateway-controller-57c96dfd77-nsdrn" Nov 04 05:37:24.478123 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:24.477946 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n49s5\" (UniqueName: \"kubernetes.io/projected/6acc41b2-6de5-4075-bbc2-78975d0b1f6a-kube-api-access-n49s5\") pod \"ai-gateway-controller-57c96dfd77-nsdrn\" (UID: \"6acc41b2-6de5-4075-bbc2-78975d0b1f6a\") " pod="opendatahub/ai-gateway-controller-57c96dfd77-nsdrn" Nov 04 05:37:24.480493 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:24.480424 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6acc41b2-6de5-4075-bbc2-78975d0b1f6a-metrics-certs\") pod \"ai-gateway-controller-57c96dfd77-nsdrn\" (UID: \"6acc41b2-6de5-4075-bbc2-78975d0b1f6a\") " pod="opendatahub/ai-gateway-controller-57c96dfd77-nsdrn" Nov 04 05:37:24.540910 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:24.540867 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n49s5\" (UniqueName: \"kubernetes.io/projected/6acc41b2-6de5-4075-bbc2-78975d0b1f6a-kube-api-access-n49s5\") pod \"ai-gateway-controller-57c96dfd77-nsdrn\" (UID: \"6acc41b2-6de5-4075-bbc2-78975d0b1f6a\") " pod="opendatahub/ai-gateway-controller-57c96dfd77-nsdrn" Nov 04 05:37:24.543770 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:24.543741 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-57c96dfd77-nsdrn" Nov 04 05:37:24.926755 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:24.926713 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-57c96dfd77-nsdrn"] Nov 04 05:37:24.931253 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:37:24.931227 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6acc41b2_6de5_4075_bbc2_78975d0b1f6a.slice/crio-518df5bdadaebf520b5532ccef2c4f852842191bbc2fbeb7f63123526fc1669d WatchSource:0}: Error finding container 518df5bdadaebf520b5532ccef2c4f852842191bbc2fbeb7f63123526fc1669d: Status 404 returned error can't find the container with id 518df5bdadaebf520b5532ccef2c4f852842191bbc2fbeb7f63123526fc1669d Nov 04 05:37:25.877737 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:25.877703 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-57c96dfd77-nsdrn" event={"ID":"6acc41b2-6de5-4075-bbc2-78975d0b1f6a","Type":"ContainerStarted","Data":"518df5bdadaebf520b5532ccef2c4f852842191bbc2fbeb7f63123526fc1669d"} Nov 04 05:37:28.894578 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:28.894546 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-57c96dfd77-nsdrn" event={"ID":"6acc41b2-6de5-4075-bbc2-78975d0b1f6a","Type":"ContainerStarted","Data":"788acc8c325947182fc973fdb1b2301dc946f314bdefb8538299253f131f7b52"} Nov 04 05:37:28.894955 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:28.894655 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-57c96dfd77-nsdrn" Nov 04 05:37:29.015299 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:29.015245 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-57c96dfd77-nsdrn" podStartSLOduration=1.8382035970000001 podStartE2EDuration="5.015230182s" totalImagesPullingTime="3.177026585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:37:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:37:29.013679488 +0000 UTC m=+582.893071972" watchObservedRunningTime="2026-11-04 05:37:29.015230182 +0000 UTC m=+582.894622654" Nov 04 05:37:30.856427 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:30.856399 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bb9489676-4rp6q" Nov 04 05:37:33.330687 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.330652 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch"] Nov 04 05:37:33.333355 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.333339 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.446837 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.446806 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch"] Nov 04 05:37:33.472304 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.472274 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plfp4\" (UniqueName: \"kubernetes.io/projected/5a97e4c0-4705-445b-9fef-ef0e282cddde-kube-api-access-plfp4\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.472465 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.472311 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.472465 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.472329 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.472465 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.472404 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.472591 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.472543 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.472784 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.472593 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.479255 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.473029 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.479255 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.473153 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.479255 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.473198 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a97e4c0-4705-445b-9fef-ef0e282cddde-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.574804 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.574762 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.574946 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.574822 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.574946 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.574864 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.574946 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.574894 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.575143 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.574953 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.575143 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.574988 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a97e4c0-4705-445b-9fef-ef0e282cddde-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.575143 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.575110 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plfp4\" (UniqueName: \"kubernetes.io/projected/5a97e4c0-4705-445b-9fef-ef0e282cddde-kube-api-access-plfp4\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.575292 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.575147 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.575292 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.575172 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.575292 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.575247 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-workload-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.575292 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.575276 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-workload-certs\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.575498 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.575325 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-data\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.575498 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.575431 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-credential-socket\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.575873 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.575852 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a97e4c0-4705-445b-9fef-ef0e282cddde-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.577414 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.577389 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-envoy\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.577563 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.577545 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-podinfo\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.641991 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.641962 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-token\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.642176 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.642107 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plfp4\" (UniqueName: \"kubernetes.io/projected/5a97e4c0-4705-445b-9fef-ef0e282cddde-kube-api-access-plfp4\") pod \"maas-default-gateway-openshift-default-778b7b5c47-xbdch\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:33.644970 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:33.644949 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:34.092718 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:34.092646 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch"] Nov 04 05:37:34.096423 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:37:34.096391 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a97e4c0_4705_445b_9fef_ef0e282cddde.slice/crio-8c9f3985e1290fb33b6b25d3b99667b2d086604d9b9394b1ef26cb432bfbd3c5 WatchSource:0}: Error finding container 8c9f3985e1290fb33b6b25d3b99667b2d086604d9b9394b1ef26cb432bfbd3c5: Status 404 returned error can't find the container with id 8c9f3985e1290fb33b6b25d3b99667b2d086604d9b9394b1ef26cb432bfbd3c5 Nov 04 05:37:34.098531 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:34.098498 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:37:34.098599 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:34.098565 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:37:34.098642 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:34.098596 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:37:34.923024 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:34.922985 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" event={"ID":"5a97e4c0-4705-445b-9fef-ef0e282cddde","Type":"ContainerStarted","Data":"14315c5cbe7073d9cfdc05599c60a018dea7802037ae3e9b31ba174b1d62d62f"} Nov 04 05:37:34.923505 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:34.923071 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" event={"ID":"5a97e4c0-4705-445b-9fef-ef0e282cddde","Type":"ContainerStarted","Data":"8c9f3985e1290fb33b6b25d3b99667b2d086604d9b9394b1ef26cb432bfbd3c5"} Nov 04 05:37:35.127723 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:35.127663 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" podStartSLOduration=2.127646166 podStartE2EDuration="2.127646166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:37:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:37:35.127367883 +0000 UTC m=+589.006760357" watchObservedRunningTime="2026-11-04 05:37:35.127646166 +0000 UTC m=+589.007038642" Nov 04 05:37:35.652099 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:35.652058 5258 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xbdch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Nov 04 05:37:35.652310 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:35.652125 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" podUID="5a97e4c0-4705-445b-9fef-ef0e282cddde" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Nov 04 05:37:35.652743 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:35.652721 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:36.657059 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:36.653441 5258 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xbdch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Nov 04 05:37:36.657059 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:36.653504 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" podUID="5a97e4c0-4705-445b-9fef-ef0e282cddde" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Nov 04 05:37:37.646067 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:37.645697 5258 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xbdch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Nov 04 05:37:37.646067 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:37.645755 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" podUID="5a97e4c0-4705-445b-9fef-ef0e282cddde" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Nov 04 05:37:38.655058 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:38.650175 5258 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xbdch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Nov 04 05:37:38.655058 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:38.650251 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" podUID="5a97e4c0-4705-445b-9fef-ef0e282cddde" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Nov 04 05:37:39.652061 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:39.648166 5258 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xbdch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Nov 04 05:37:39.652061 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:39.648236 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" podUID="5a97e4c0-4705-445b-9fef-ef0e282cddde" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Nov 04 05:37:39.900987 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:39.900955 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-57c96dfd77-nsdrn" Nov 04 05:37:40.645991 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:40.645958 5258 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xbdch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Nov 04 05:37:40.646220 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:40.646008 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" podUID="5a97e4c0-4705-445b-9fef-ef0e282cddde" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Nov 04 05:37:41.645781 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:41.645742 5258 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-778b7b5c47-xbdch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Nov 04 05:37:41.646272 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:41.645801 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" podUID="5a97e4c0-4705-445b-9fef-ef0e282cddde" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Nov 04 05:37:42.653749 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:42.653714 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:42.654486 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:42.654061 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:42.654853 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:37:42.654831 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:37:52.780808 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:37:52.780767 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:38:22.780362 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:38:22.780328 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:38:36.081074 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.080985 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm"] Nov 04 05:38:36.083797 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.083778 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.143891 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.143857 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-a84d48-openshift-default-dockercfg-cqcsm\"" Nov 04 05:38:36.176654 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.176620 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-token\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.176811 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.176682 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-credential-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.176811 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.176766 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-workload-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.176811 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.176808 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-workload-certs\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.176954 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.176927 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb4gj\" (UniqueName: \"kubernetes.io/projected/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-kube-api-access-hb4gj\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.177011 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.176988 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-envoy\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.177087 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.177057 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-data\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.177139 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.177094 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-podinfo\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.177183 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.177162 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istiod-ca-cert\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.201759 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.201728 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm"] Nov 04 05:38:36.277654 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.277626 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istiod-ca-cert\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.277848 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.277680 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-token\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.277848 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.277712 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-credential-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.277848 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.277804 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-workload-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.278267 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.277847 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-workload-certs\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.278267 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.277900 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb4gj\" (UniqueName: \"kubernetes.io/projected/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-kube-api-access-hb4gj\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.278267 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.277942 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-envoy\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.278267 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.277967 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-data\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.278267 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.277995 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-podinfo\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.278524 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.278271 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-credential-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.278524 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.278368 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-data\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.278524 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.278447 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istiod-ca-cert\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.278524 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.278483 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-workload-certs\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.278730 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.278650 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-workload-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.280213 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.280191 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-envoy\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.280589 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.280570 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-podinfo\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.347519 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.347447 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-token\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.347711 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.347687 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb4gj\" (UniqueName: \"kubernetes.io/projected/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-kube-api-access-hb4gj\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.395278 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.395243 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:36.410891 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.410859 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4"] Nov 04 05:38:36.413470 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.413449 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.480278 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.480249 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnrlh\" (UniqueName: \"kubernetes.io/projected/86f487ad-d8ba-40ca-9f08-b4f560862516-kube-api-access-qnrlh\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.480648 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.480577 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-credential-socket\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.480648 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.480625 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-data\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.480818 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.480695 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86f487ad-d8ba-40ca-9f08-b4f560862516-istiod-ca-cert\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.480909 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.480867 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-token\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.481050 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.481009 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-envoy\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.481132 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.481087 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-podinfo\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.481189 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.481158 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-workload-socket\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.481246 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.481187 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-workload-certs\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.483842 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.483818 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4"] Nov 04 05:38:36.553553 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.553515 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-fbbad9-openshift-default-dockercfg-nb2h5\"" Nov 04 05:38:36.581892 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.581863 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86f487ad-d8ba-40ca-9f08-b4f560862516-istiod-ca-cert\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.582059 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.581919 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-token\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.582059 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.581950 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-envoy\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.582059 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.581967 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-podinfo\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.582059 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.581995 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-workload-socket\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.582059 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.582012 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-workload-certs\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.582327 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.582065 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnrlh\" (UniqueName: \"kubernetes.io/projected/86f487ad-d8ba-40ca-9f08-b4f560862516-kube-api-access-qnrlh\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.582327 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.582116 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-credential-socket\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.582327 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.582141 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-data\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.582609 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.582580 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-workload-socket\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.582730 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.582639 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-credential-socket\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.582730 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.582673 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-workload-certs\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.582730 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.582685 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86f487ad-d8ba-40ca-9f08-b4f560862516-istiod-ca-cert\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.582845 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.582749 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-data\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.584434 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.584415 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-envoy\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.584507 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.584484 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-podinfo\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.650235 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.650209 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-token\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.650452 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.650429 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnrlh\" (UniqueName: \"kubernetes.io/projected/86f487ad-d8ba-40ca-9f08-b4f560862516-kube-api-access-qnrlh\") pod \"e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.728663 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.728634 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:36.843589 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.843560 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm"] Nov 04 05:38:36.847342 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:38:36.847311 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a3d0b5a_4e73_4939_97f5_f662b5bc6b15.slice/crio-3bfff75a2389fb89d9813cc6ae58f03c4798532c616c810d91b2dd17b729001d WatchSource:0}: Error finding container 3bfff75a2389fb89d9813cc6ae58f03c4798532c616c810d91b2dd17b729001d: Status 404 returned error can't find the container with id 3bfff75a2389fb89d9813cc6ae58f03c4798532c616c810d91b2dd17b729001d Nov 04 05:38:36.849605 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.849576 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:38:36.849669 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.849636 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:38:36.849669 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:36.849664 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:38:37.122410 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.122376 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4"] Nov 04 05:38:37.126395 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:38:37.126363 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86f487ad_d8ba_40ca_9f08_b4f560862516.slice/crio-0ab9455c82362cdebf55165c1407500c8c33bf4499f0526314653e3daeab7026 WatchSource:0}: Error finding container 0ab9455c82362cdebf55165c1407500c8c33bf4499f0526314653e3daeab7026: Status 404 returned error can't find the container with id 0ab9455c82362cdebf55165c1407500c8c33bf4499f0526314653e3daeab7026 Nov 04 05:38:37.128637 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.128607 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:38:37.128726 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.128671 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:38:37.128726 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.128702 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:38:37.236891 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.236836 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" event={"ID":"86f487ad-d8ba-40ca-9f08-b4f560862516","Type":"ContainerStarted","Data":"8fc01b5d1790f5726d1ba0e0eece5ac03a9f09e1c4bbea055951536672444cee"} Nov 04 05:38:37.237129 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.236913 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" event={"ID":"86f487ad-d8ba-40ca-9f08-b4f560862516","Type":"ContainerStarted","Data":"0ab9455c82362cdebf55165c1407500c8c33bf4499f0526314653e3daeab7026"} Nov 04 05:38:37.240575 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.239795 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" event={"ID":"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15","Type":"ContainerStarted","Data":"79e87858931c3487bd37db62c4fef611d201642dbf9ad792ba351318b73991a8"} Nov 04 05:38:37.240575 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.239856 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" event={"ID":"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15","Type":"ContainerStarted","Data":"3bfff75a2389fb89d9813cc6ae58f03c4798532c616c810d91b2dd17b729001d"} Nov 04 05:38:37.377605 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.377515 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" podStartSLOduration=1.377498226 podStartE2EDuration="1.377498226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:38:37.376000545 +0000 UTC m=+651.255393024" watchObservedRunningTime="2026-11-04 05:38:37.377498226 +0000 UTC m=+651.256890699" Nov 04 05:38:37.380641 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.380613 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd"] Nov 04 05:38:37.383431 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.383411 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.395572 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.395547 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:37.400431 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.400414 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:37.442826 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.442740 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-283815-openshift-default-dockercfg-d64tn\"" Nov 04 05:38:37.451904 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.451868 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd"] Nov 04 05:38:37.490818 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.490783 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-envoy\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.491008 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.490838 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-workload-socket\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.491008 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.490895 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nhfs\" (UniqueName: \"kubernetes.io/projected/5cfab36f-9d70-4efa-96ed-a0db577b9939-kube-api-access-6nhfs\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.491008 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.490981 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-podinfo\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.491285 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.491052 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-credential-socket\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.491285 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.491091 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cfab36f-9d70-4efa-96ed-a0db577b9939-istiod-ca-cert\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.491285 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.491115 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-data\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.491285 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.491180 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-token\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.491285 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.491203 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-workload-certs\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.512778 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.512108 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" podStartSLOduration=1.5120875150000002 podStartE2EDuration="1.512087515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:38:37.510415096 +0000 UTC m=+651.389807569" watchObservedRunningTime="2026-11-04 05:38:37.512087515 +0000 UTC m=+651.391479989" Nov 04 05:38:37.592579 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.592542 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-token\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.592579 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.592580 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-workload-certs\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.592784 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.592761 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-envoy\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.592831 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.592817 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-workload-socket\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.592867 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.592837 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nhfs\" (UniqueName: \"kubernetes.io/projected/5cfab36f-9d70-4efa-96ed-a0db577b9939-kube-api-access-6nhfs\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.592911 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.592884 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-podinfo\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.592968 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.592953 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-credential-socket\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.593028 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.592986 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cfab36f-9d70-4efa-96ed-a0db577b9939-istiod-ca-cert\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.593028 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.593007 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-data\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.593173 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.593067 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-workload-certs\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.593173 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.593156 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-workload-socket\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.593431 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.593404 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-credential-socket\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.593431 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.593425 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-data\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.593784 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.593766 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cfab36f-9d70-4efa-96ed-a0db577b9939-istiod-ca-cert\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.595092 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.595065 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-envoy\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.595381 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.595365 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-podinfo\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.710471 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.710396 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-token\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.711589 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.711565 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nhfs\" (UniqueName: \"kubernetes.io/projected/5cfab36f-9d70-4efa-96ed-a0db577b9939-kube-api-access-6nhfs\") pod \"e2e-worker-w5-283815-openshift-default-578d454689-qw4rd\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:37.729178 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.729153 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:37.996313 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:37.996231 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:38.147379 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.147347 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw"] Nov 04 05:38:38.150057 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.150040 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.221665 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.221629 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw"] Nov 04 05:38:38.228807 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.228772 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-961485-openshift-default-dockercfg-qhhl7\"" Nov 04 05:38:38.244059 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.244011 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:38.245009 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.244990 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:38:38.301432 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.301343 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf234545-c654-43c2-b9ee-a023302087f8-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.301606 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.301485 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.301606 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.301527 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf234545-c654-43c2-b9ee-a023302087f8-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.301606 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.301572 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.301606 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.301597 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.301827 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.301641 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf234545-c654-43c2-b9ee-a023302087f8-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.301827 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.301754 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.301934 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.301920 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.302017 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.302004 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj2c9\" (UniqueName: \"kubernetes.io/projected/cf234545-c654-43c2-b9ee-a023302087f8-kube-api-access-rj2c9\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.402966 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.402924 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.403202 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.403173 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj2c9\" (UniqueName: \"kubernetes.io/projected/cf234545-c654-43c2-b9ee-a023302087f8-kube-api-access-rj2c9\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.403287 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.403272 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf234545-c654-43c2-b9ee-a023302087f8-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.403346 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.403308 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.403404 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.403373 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.403460 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.403419 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf234545-c654-43c2-b9ee-a023302087f8-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.403512 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.403492 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.403772 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.403741 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.403872 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.403845 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf234545-c654-43c2-b9ee-a023302087f8-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.403941 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.403883 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.403997 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.403959 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.404081 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.403987 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf234545-c654-43c2-b9ee-a023302087f8-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.404081 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.404053 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.404261 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.404238 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.405538 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.405515 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.406072 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.406056 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf234545-c654-43c2-b9ee-a023302087f8-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.443794 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.443763 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd"] Nov 04 05:38:38.447450 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:38:38.447417 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cfab36f_9d70_4efa_96ed_a0db577b9939.slice/crio-1861dd910c25c39c3630ced8311e40566680a6e825329fb22f42a2989c2f50c7 WatchSource:0}: Error finding container 1861dd910c25c39c3630ced8311e40566680a6e825329fb22f42a2989c2f50c7: Status 404 returned error can't find the container with id 1861dd910c25c39c3630ced8311e40566680a6e825329fb22f42a2989c2f50c7 Nov 04 05:38:38.449994 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.449965 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:38:38.450131 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.450065 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:38:38.450131 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.450109 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:38:38.476483 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.476451 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf234545-c654-43c2-b9ee-a023302087f8-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.476583 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.476568 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj2c9\" (UniqueName: \"kubernetes.io/projected/cf234545-c654-43c2-b9ee-a023302087f8-kube-api-access-rj2c9\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:38.735752 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.735719 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:38.762359 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:38.762328 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:39.153450 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:39.153412 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw"] Nov 04 05:38:39.161991 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:39.161958 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:38:39.162114 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:39.162077 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:38:39.162183 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:39.162130 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:38:39.250643 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:39.250608 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" event={"ID":"cf234545-c654-43c2-b9ee-a023302087f8","Type":"ContainerStarted","Data":"f190f45506b3f8f2b972d0bdfaea00beee8b19df0398064344cda34e657d346a"} Nov 04 05:38:39.250815 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:39.250675 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" event={"ID":"cf234545-c654-43c2-b9ee-a023302087f8","Type":"ContainerStarted","Data":"73962c7d2ce60854391aa694d1a92dd6ca854716e01d5da7fe8a94ebd653862c"} Nov 04 05:38:39.252294 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:39.252261 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" event={"ID":"5cfab36f-9d70-4efa-96ed-a0db577b9939","Type":"ContainerStarted","Data":"1b0184779bba0f5c00fe4c33fc7bb8c68edb0ad56d17c138302fd4282a345935"} Nov 04 05:38:39.252453 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:39.252303 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" event={"ID":"5cfab36f-9d70-4efa-96ed-a0db577b9939","Type":"ContainerStarted","Data":"1861dd910c25c39c3630ced8311e40566680a6e825329fb22f42a2989c2f50c7"} Nov 04 05:38:39.252674 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:39.252651 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:39.253587 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:39.253566 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:38:39.390157 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:39.390103 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" podStartSLOduration=1.390088434 podStartE2EDuration="1.390088434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:38:39.38926742 +0000 UTC m=+653.268659894" watchObservedRunningTime="2026-11-04 05:38:39.390088434 +0000 UTC m=+653.269480904" Nov 04 05:38:39.667172 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:39.667122 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" podStartSLOduration=2.667107104 podStartE2EDuration="2.667107104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:38:39.665907187 +0000 UTC m=+653.545299679" watchObservedRunningTime="2026-11-04 05:38:39.667107104 +0000 UTC m=+653.546499576" Nov 04 05:38:39.763077 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:39.763023 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:39.997265 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:39.997179 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:40.002574 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:40.002548 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:40.257382 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:40.257303 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:40.258182 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:40.258159 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:38:40.768273 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:40.768246 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:41.261963 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:41.261929 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:41.262847 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:41.262824 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:38:46.501770 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:46.501737 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx"] Nov 04 05:38:46.507069 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:46.507028 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx" Nov 04 05:38:46.597349 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:46.597314 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx\" (UID: \"c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx" Nov 04 05:38:46.597512 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:46.597443 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psxfq\" (UniqueName: \"kubernetes.io/projected/c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1-kube-api-access-psxfq\") pod \"payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx\" (UID: \"c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx" Nov 04 05:38:46.623140 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:46.623108 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-fbbad9-dockercfg-d4s4t\"" Nov 04 05:38:46.623140 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:46.623108 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-fbbad9\"" Nov 04 05:38:46.625283 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:46.625254 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx"] Nov 04 05:38:46.698473 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:46.698442 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psxfq\" (UniqueName: \"kubernetes.io/projected/c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1-kube-api-access-psxfq\") pod \"payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx\" (UID: \"c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx" Nov 04 05:38:46.698655 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:46.698536 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx\" (UID: \"c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx" Nov 04 05:38:46.699266 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:46.699246 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1-plugins-config-volume\") pod \"payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx\" (UID: \"c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx" Nov 04 05:38:46.768103 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:46.768021 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psxfq\" (UniqueName: \"kubernetes.io/projected/c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1-kube-api-access-psxfq\") pod \"payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx\" (UID: \"c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1\") " pod="openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx" Nov 04 05:38:46.820914 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:46.820888 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx" Nov 04 05:38:47.219055 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:47.218928 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx"] Nov 04 05:38:47.234769 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:38:47.234715 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5196a26_04e3_41d2_ab3a_0e8e5dbd01f1.slice/crio-451a67058dd80233b3f7d8ba1812bf08a2cf09c3cbdd6542e788e940e1ecb365 WatchSource:0}: Error finding container 451a67058dd80233b3f7d8ba1812bf08a2cf09c3cbdd6542e788e940e1ecb365: Status 404 returned error can't find the container with id 451a67058dd80233b3f7d8ba1812bf08a2cf09c3cbdd6542e788e940e1ecb365 Nov 04 05:38:47.291441 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:47.291414 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx" event={"ID":"c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1","Type":"ContainerStarted","Data":"451a67058dd80233b3f7d8ba1812bf08a2cf09c3cbdd6542e788e940e1ecb365"} Nov 04 05:38:48.298459 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:48.298418 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx" event={"ID":"c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1","Type":"ContainerStarted","Data":"6462374c1c0fa5d9b51e03551d12db632adad4541d30cb5e064c34dae6804cf0"} Nov 04 05:38:48.298864 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:48.298508 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx" Nov 04 05:38:48.428600 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:48.428548 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx" podStartSLOduration=2.428533857 podStartE2EDuration="2.428533857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:38:48.42655742 +0000 UTC m=+662.305949894" watchObservedRunningTime="2026-11-04 05:38:48.428533857 +0000 UTC m=+662.307926330" Nov 04 05:38:52.780674 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:38:52.780635 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:38:54.370108 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:54.370075 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m"] Nov 04 05:38:54.372900 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:54.372883 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m" Nov 04 05:38:54.484574 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:54.483836 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndc4m\" (UniqueName: \"kubernetes.io/projected/37be0c15-5de0-4b8f-aaa3-5bbe4c186f49-kube-api-access-ndc4m\") pod \"payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m\" (UID: \"37be0c15-5de0-4b8f-aaa3-5bbe4c186f49\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m" Nov 04 05:38:54.484574 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:54.484247 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37be0c15-5de0-4b8f-aaa3-5bbe4c186f49-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m\" (UID: \"37be0c15-5de0-4b8f-aaa3-5bbe4c186f49\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m" Nov 04 05:38:54.488024 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:54.487977 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-098178-dockercfg-2xgq6\"" Nov 04 05:38:54.488918 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:54.488854 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-098178\"" Nov 04 05:38:54.492474 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:54.492451 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m"] Nov 04 05:38:54.550225 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:54.546873 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm"] Nov 04 05:38:54.554195 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:54.554169 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm" Nov 04 05:38:54.584895 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:54.584863 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37be0c15-5de0-4b8f-aaa3-5bbe4c186f49-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m\" (UID: \"37be0c15-5de0-4b8f-aaa3-5bbe4c186f49\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m" Nov 04 05:38:54.585018 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:54.584918 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpqf6\" (UniqueName: \"kubernetes.io/projected/3552c62c-8a98-4845-9817-16b284233c6f-kube-api-access-vpqf6\") pod \"payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm\" (UID: \"3552c62c-8a98-4845-9817-16b284233c6f\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm" Nov 04 05:38:54.585018 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:54.584947 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndc4m\" (UniqueName: \"kubernetes.io/projected/37be0c15-5de0-4b8f-aaa3-5bbe4c186f49-kube-api-access-ndc4m\") pod \"payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m\" (UID: \"37be0c15-5de0-4b8f-aaa3-5bbe4c186f49\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m" Nov 04 05:38:54.585131 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:54.585115 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3552c62c-8a98-4845-9817-16b284233c6f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm\" (UID: \"3552c62c-8a98-4845-9817-16b284233c6f\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm" Nov 04 05:38:54.585520 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:54.585503 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37be0c15-5de0-4b8f-aaa3-5bbe4c186f49-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m\" (UID: \"37be0c15-5de0-4b8f-aaa3-5bbe4c186f49\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m" Nov 04 05:38:54.597418 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:54.597390 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm"] Nov 04 05:38:54.653549 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:54.653471 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndc4m\" (UniqueName: \"kubernetes.io/projected/37be0c15-5de0-4b8f-aaa3-5bbe4c186f49-kube-api-access-ndc4m\") pod \"payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m\" (UID: \"37be0c15-5de0-4b8f-aaa3-5bbe4c186f49\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m" Nov 04 05:38:54.684992 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:54.684961 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m" Nov 04 05:38:54.685802 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:54.685770 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3552c62c-8a98-4845-9817-16b284233c6f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm\" (UID: \"3552c62c-8a98-4845-9817-16b284233c6f\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm" Nov 04 05:38:54.685919 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:54.685860 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpqf6\" (UniqueName: \"kubernetes.io/projected/3552c62c-8a98-4845-9817-16b284233c6f-kube-api-access-vpqf6\") pod \"payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm\" (UID: \"3552c62c-8a98-4845-9817-16b284233c6f\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm" Nov 04 05:38:54.686471 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:54.686449 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3552c62c-8a98-4845-9817-16b284233c6f-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm\" (UID: \"3552c62c-8a98-4845-9817-16b284233c6f\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm" Nov 04 05:38:54.755639 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:54.755601 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpqf6\" (UniqueName: \"kubernetes.io/projected/3552c62c-8a98-4845-9817-16b284233c6f-kube-api-access-vpqf6\") pod \"payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm\" (UID: \"3552c62c-8a98-4845-9817-16b284233c6f\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm" Nov 04 05:38:54.866159 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:54.866129 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm" Nov 04 05:38:55.072398 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:55.072369 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m"] Nov 04 05:38:55.075517 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:38:55.075491 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37be0c15_5de0_4b8f_aaa3_5bbe4c186f49.slice/crio-c613ea9f5abb6895ae8003f7c0046d67dac5cf684b3d9ec896dc2cccfe66ab05 WatchSource:0}: Error finding container c613ea9f5abb6895ae8003f7c0046d67dac5cf684b3d9ec896dc2cccfe66ab05: Status 404 returned error can't find the container with id c613ea9f5abb6895ae8003f7c0046d67dac5cf684b3d9ec896dc2cccfe66ab05 Nov 04 05:38:55.250385 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:55.250307 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm"] Nov 04 05:38:55.253264 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:38:55.253236 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3552c62c_8a98_4845_9817_16b284233c6f.slice/crio-77078d38b1fe9daea57779971d37dd95a6ea3a70bcffde6558fa4d18e3addd28 WatchSource:0}: Error finding container 77078d38b1fe9daea57779971d37dd95a6ea3a70bcffde6558fa4d18e3addd28: Status 404 returned error can't find the container with id 77078d38b1fe9daea57779971d37dd95a6ea3a70bcffde6558fa4d18e3addd28 Nov 04 05:38:55.333636 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:55.333600 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm" event={"ID":"3552c62c-8a98-4845-9817-16b284233c6f","Type":"ContainerStarted","Data":"2ce5bf89b5e04041e69560a12f888374946167005c70d20158dd26c2b96d422b"} Nov 04 05:38:55.333788 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:55.333661 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm" event={"ID":"3552c62c-8a98-4845-9817-16b284233c6f","Type":"ContainerStarted","Data":"77078d38b1fe9daea57779971d37dd95a6ea3a70bcffde6558fa4d18e3addd28"} Nov 04 05:38:55.333874 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:55.333784 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm" Nov 04 05:38:55.335094 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:55.335074 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m" event={"ID":"37be0c15-5de0-4b8f-aaa3-5bbe4c186f49","Type":"ContainerStarted","Data":"33d8d1552f01e251bdc8a1952171fec4fdabc0457f3bead08acb118ce23691cf"} Nov 04 05:38:55.335210 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:55.335107 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m" event={"ID":"37be0c15-5de0-4b8f-aaa3-5bbe4c186f49","Type":"ContainerStarted","Data":"c613ea9f5abb6895ae8003f7c0046d67dac5cf684b3d9ec896dc2cccfe66ab05"} Nov 04 05:38:55.335210 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:55.335128 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m" Nov 04 05:38:55.464896 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:55.464832 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm" podStartSLOduration=1.464817941 podStartE2EDuration="1.464817941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:38:55.463420985 +0000 UTC m=+669.342813469" watchObservedRunningTime="2026-11-04 05:38:55.464817941 +0000 UTC m=+669.344210455" Nov 04 05:38:55.592327 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:55.592212 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m" podStartSLOduration=1.5921974859999999 podStartE2EDuration="1.592197486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:38:55.591584947 +0000 UTC m=+669.470977494" watchObservedRunningTime="2026-11-04 05:38:55.592197486 +0000 UTC m=+669.471589959" Nov 04 05:38:56.909783 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:56.909740 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:38:59.306491 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:59.306463 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx" Nov 04 05:38:59.773975 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:59.773940 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt"] Nov 04 05:38:59.776711 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:59.776694 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt" Nov 04 05:38:59.854057 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:59.853204 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-90dc8d\"" Nov 04 05:38:59.854057 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:59.853961 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-90dc8d-dockercfg-tm9w6\"" Nov 04 05:38:59.858055 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:59.855434 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt"] Nov 04 05:38:59.858688 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:59.858658 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/775cd728-84a4-43a3-b3a1-2fdcbb19203e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt\" (UID: \"775cd728-84a4-43a3-b3a1-2fdcbb19203e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt" Nov 04 05:38:59.858963 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:59.858940 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m72bm\" (UniqueName: \"kubernetes.io/projected/775cd728-84a4-43a3-b3a1-2fdcbb19203e-kube-api-access-m72bm\") pod \"payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt\" (UID: \"775cd728-84a4-43a3-b3a1-2fdcbb19203e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt" Nov 04 05:38:59.904885 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:59.904845 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn"] Nov 04 05:38:59.914652 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:59.914621 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn" Nov 04 05:38:59.960995 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:59.960960 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6634277-09ef-4074-9aab-36e8e10155d5-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn\" (UID: \"a6634277-09ef-4074-9aab-36e8e10155d5\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn" Nov 04 05:38:59.961274 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:59.961018 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m72bm\" (UniqueName: \"kubernetes.io/projected/775cd728-84a4-43a3-b3a1-2fdcbb19203e-kube-api-access-m72bm\") pod \"payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt\" (UID: \"775cd728-84a4-43a3-b3a1-2fdcbb19203e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt" Nov 04 05:38:59.961274 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:59.961142 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvgbh\" (UniqueName: \"kubernetes.io/projected/a6634277-09ef-4074-9aab-36e8e10155d5-kube-api-access-zvgbh\") pod \"payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn\" (UID: \"a6634277-09ef-4074-9aab-36e8e10155d5\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn" Nov 04 05:38:59.961274 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:59.961243 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/775cd728-84a4-43a3-b3a1-2fdcbb19203e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt\" (UID: \"775cd728-84a4-43a3-b3a1-2fdcbb19203e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt" Nov 04 05:38:59.962317 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:59.962295 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/775cd728-84a4-43a3-b3a1-2fdcbb19203e-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt\" (UID: \"775cd728-84a4-43a3-b3a1-2fdcbb19203e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt" Nov 04 05:38:59.971511 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:38:59.971475 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn"] Nov 04 05:39:00.030241 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:00.030167 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m72bm\" (UniqueName: \"kubernetes.io/projected/775cd728-84a4-43a3-b3a1-2fdcbb19203e-kube-api-access-m72bm\") pod \"payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt\" (UID: \"775cd728-84a4-43a3-b3a1-2fdcbb19203e\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt" Nov 04 05:39:00.062390 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:00.062354 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6634277-09ef-4074-9aab-36e8e10155d5-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn\" (UID: \"a6634277-09ef-4074-9aab-36e8e10155d5\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn" Nov 04 05:39:00.062679 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:00.062656 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvgbh\" (UniqueName: \"kubernetes.io/projected/a6634277-09ef-4074-9aab-36e8e10155d5-kube-api-access-zvgbh\") pod \"payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn\" (UID: \"a6634277-09ef-4074-9aab-36e8e10155d5\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn" Nov 04 05:39:00.063979 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:00.063954 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6634277-09ef-4074-9aab-36e8e10155d5-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn\" (UID: \"a6634277-09ef-4074-9aab-36e8e10155d5\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn" Nov 04 05:39:00.089613 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:00.089579 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt" Nov 04 05:39:00.130717 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:00.130684 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvgbh\" (UniqueName: \"kubernetes.io/projected/a6634277-09ef-4074-9aab-36e8e10155d5-kube-api-access-zvgbh\") pod \"payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn\" (UID: \"a6634277-09ef-4074-9aab-36e8e10155d5\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn" Nov 04 05:39:00.257510 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:00.257223 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn" Nov 04 05:39:00.502676 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:00.502644 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt"] Nov 04 05:39:00.508865 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:39:00.508825 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod775cd728_84a4_43a3_b3a1_2fdcbb19203e.slice/crio-196c03d2338ed97f9fbc029cada170040ed78c93e2b2d9939e5e6f1c8b271666 WatchSource:0}: Error finding container 196c03d2338ed97f9fbc029cada170040ed78c93e2b2d9939e5e6f1c8b271666: Status 404 returned error can't find the container with id 196c03d2338ed97f9fbc029cada170040ed78c93e2b2d9939e5e6f1c8b271666 Nov 04 05:39:00.652059 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:00.645425 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn"] Nov 04 05:39:00.666446 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:39:00.666411 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6634277_09ef_4074_9aab_36e8e10155d5.slice/crio-472d2ca920781808f6d4fc9db5f6410bfb0de70c9fe04e40080a5255da81276e WatchSource:0}: Error finding container 472d2ca920781808f6d4fc9db5f6410bfb0de70c9fe04e40080a5255da81276e: Status 404 returned error can't find the container with id 472d2ca920781808f6d4fc9db5f6410bfb0de70c9fe04e40080a5255da81276e Nov 04 05:39:01.386787 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:01.386755 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn" event={"ID":"a6634277-09ef-4074-9aab-36e8e10155d5","Type":"ContainerStarted","Data":"a3d905b2155cb14412a920bfa51a8a97b78827e114f0799aa3cf1d71af491f9e"} Nov 04 05:39:01.387075 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:01.387056 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn" Nov 04 05:39:01.387185 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:01.387171 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn" event={"ID":"a6634277-09ef-4074-9aab-36e8e10155d5","Type":"ContainerStarted","Data":"472d2ca920781808f6d4fc9db5f6410bfb0de70c9fe04e40080a5255da81276e"} Nov 04 05:39:01.388203 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:01.388184 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt" event={"ID":"775cd728-84a4-43a3-b3a1-2fdcbb19203e","Type":"ContainerStarted","Data":"1f86f6ed357fdfcdfc354387501d04f8982eef96b928ba591fc7234b01f42d0b"} Nov 04 05:39:01.388319 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:01.388308 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt" event={"ID":"775cd728-84a4-43a3-b3a1-2fdcbb19203e","Type":"ContainerStarted","Data":"196c03d2338ed97f9fbc029cada170040ed78c93e2b2d9939e5e6f1c8b271666"} Nov 04 05:39:01.388389 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:01.388381 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt" Nov 04 05:39:01.529868 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:01.529808 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn" podStartSLOduration=2.52979449 podStartE2EDuration="2.52979449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:39:01.5285662 +0000 UTC m=+675.407958674" watchObservedRunningTime="2026-11-04 05:39:01.52979449 +0000 UTC m=+675.409186962" Nov 04 05:39:01.668523 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:01.667813 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt" podStartSLOduration=2.667790827 podStartE2EDuration="2.667790827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:38:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:39:01.666525107 +0000 UTC m=+675.545917581" watchObservedRunningTime="2026-11-04 05:39:01.667790827 +0000 UTC m=+675.547183301" Nov 04 05:39:05.118639 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.118605 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf"] Nov 04 05:39:05.121665 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.121643 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf" Nov 04 05:39:05.181447 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.181416 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-901a00\"" Nov 04 05:39:05.218511 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.218466 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8bjc\" (UniqueName: \"kubernetes.io/projected/b1f775db-6655-46cf-8283-e25689017416-kube-api-access-r8bjc\") pod \"payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf\" (UID: \"b1f775db-6655-46cf-8283-e25689017416\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf" Nov 04 05:39:05.218682 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.218662 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1f775db-6655-46cf-8283-e25689017416-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf\" (UID: \"b1f775db-6655-46cf-8283-e25689017416\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf" Nov 04 05:39:05.236616 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.236591 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf"] Nov 04 05:39:05.267802 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.267776 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k"] Nov 04 05:39:05.270554 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.270538 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k" Nov 04 05:39:05.319872 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.319830 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1f775db-6655-46cf-8283-e25689017416-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf\" (UID: \"b1f775db-6655-46cf-8283-e25689017416\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf" Nov 04 05:39:05.320746 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.320720 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8bjc\" (UniqueName: \"kubernetes.io/projected/b1f775db-6655-46cf-8283-e25689017416-kube-api-access-r8bjc\") pod \"payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf\" (UID: \"b1f775db-6655-46cf-8283-e25689017416\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf" Nov 04 05:39:05.321382 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.321361 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/572c549a-0b75-4f41-8bf5-a502b67fdf78-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k\" (UID: \"572c549a-0b75-4f41-8bf5-a502b67fdf78\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k" Nov 04 05:39:05.321580 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.321563 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5j96\" (UniqueName: \"kubernetes.io/projected/572c549a-0b75-4f41-8bf5-a502b67fdf78-kube-api-access-n5j96\") pod \"payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k\" (UID: \"572c549a-0b75-4f41-8bf5-a502b67fdf78\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k" Nov 04 05:39:05.322270 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.322139 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1f775db-6655-46cf-8283-e25689017416-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf\" (UID: \"b1f775db-6655-46cf-8283-e25689017416\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf" Nov 04 05:39:05.349234 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.349142 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k"] Nov 04 05:39:05.350592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.349992 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-901a00-dockercfg-nltmh\"" Nov 04 05:39:05.389443 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.389347 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8bjc\" (UniqueName: \"kubernetes.io/projected/b1f775db-6655-46cf-8283-e25689017416-kube-api-access-r8bjc\") pod \"payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf\" (UID: \"b1f775db-6655-46cf-8283-e25689017416\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf" Nov 04 05:39:05.422792 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.422764 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/572c549a-0b75-4f41-8bf5-a502b67fdf78-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k\" (UID: \"572c549a-0b75-4f41-8bf5-a502b67fdf78\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k" Nov 04 05:39:05.422952 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.422799 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5j96\" (UniqueName: \"kubernetes.io/projected/572c549a-0b75-4f41-8bf5-a502b67fdf78-kube-api-access-n5j96\") pod \"payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k\" (UID: \"572c549a-0b75-4f41-8bf5-a502b67fdf78\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k" Nov 04 05:39:05.423535 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.423514 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/572c549a-0b75-4f41-8bf5-a502b67fdf78-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k\" (UID: \"572c549a-0b75-4f41-8bf5-a502b67fdf78\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k" Nov 04 05:39:05.435252 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.435228 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf" Nov 04 05:39:05.494130 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.492727 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5j96\" (UniqueName: \"kubernetes.io/projected/572c549a-0b75-4f41-8bf5-a502b67fdf78-kube-api-access-n5j96\") pod \"payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k\" (UID: \"572c549a-0b75-4f41-8bf5-a502b67fdf78\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k" Nov 04 05:39:05.583477 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.583444 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k" Nov 04 05:39:05.826760 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.826684 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf"] Nov 04 05:39:05.830064 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:39:05.830025 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1f775db_6655_46cf_8283_e25689017416.slice/crio-2c805d39105686a7cd5e26ef47722b0bbe7da7999cf87805f3b09d6a3f53a662 WatchSource:0}: Error finding container 2c805d39105686a7cd5e26ef47722b0bbe7da7999cf87805f3b09d6a3f53a662: Status 404 returned error can't find the container with id 2c805d39105686a7cd5e26ef47722b0bbe7da7999cf87805f3b09d6a3f53a662 Nov 04 05:39:05.970720 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:05.970689 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k"] Nov 04 05:39:05.974517 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:39:05.974492 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod572c549a_0b75_4f41_8bf5_a502b67fdf78.slice/crio-a17047d76f7b807052116ea0c6123904a2e80fb5e6a33c1bfc76682defec1bae WatchSource:0}: Error finding container a17047d76f7b807052116ea0c6123904a2e80fb5e6a33c1bfc76682defec1bae: Status 404 returned error can't find the container with id a17047d76f7b807052116ea0c6123904a2e80fb5e6a33c1bfc76682defec1bae Nov 04 05:39:06.343189 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:06.343108 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m" Nov 04 05:39:06.343697 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:06.343258 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm" Nov 04 05:39:06.414702 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:06.414670 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf" event={"ID":"b1f775db-6655-46cf-8283-e25689017416","Type":"ContainerStarted","Data":"ab76d1530bfcf46e1c70ffdd7e3b834e70817bab17efb21527e4d74a94144bb9"} Nov 04 05:39:06.414859 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:06.414717 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf" event={"ID":"b1f775db-6655-46cf-8283-e25689017416","Type":"ContainerStarted","Data":"2c805d39105686a7cd5e26ef47722b0bbe7da7999cf87805f3b09d6a3f53a662"} Nov 04 05:39:06.414859 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:06.414767 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf" Nov 04 05:39:06.416209 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:06.416190 5258 generic.go:356] "Generic (PLEG): container finished" podID="6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" containerID="79e87858931c3487bd37db62c4fef611d201642dbf9ad792ba351318b73991a8" exitCode=0 Nov 04 05:39:06.416306 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:06.416257 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" event={"ID":"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15","Type":"ContainerDied","Data":"79e87858931c3487bd37db62c4fef611d201642dbf9ad792ba351318b73991a8"} Nov 04 05:39:06.416597 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:06.416580 5258 scope.go:122] "RemoveContainer" containerID="79e87858931c3487bd37db62c4fef611d201642dbf9ad792ba351318b73991a8" Nov 04 05:39:06.417754 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:06.417734 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k" event={"ID":"572c549a-0b75-4f41-8bf5-a502b67fdf78","Type":"ContainerStarted","Data":"6a695e4f4fa29b369eb910ed9123b7b7327f442f34506ac1ed19214c87633d56"} Nov 04 05:39:06.417846 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:06.417774 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k" event={"ID":"572c549a-0b75-4f41-8bf5-a502b67fdf78","Type":"ContainerStarted","Data":"a17047d76f7b807052116ea0c6123904a2e80fb5e6a33c1bfc76682defec1bae"} Nov 04 05:39:06.417846 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:06.417810 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k" Nov 04 05:39:06.417936 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:06.417832 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:39:06.417936 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:06.417907 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:39:06.417998 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:06.417957 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:39:06.601011 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:06.600977 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:39:06.735815 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:06.735700 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf" podStartSLOduration=1.735679872 podStartE2EDuration="1.735679872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:39:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:39:06.733812837 +0000 UTC m=+680.613205317" watchObservedRunningTime="2026-11-04 05:39:06.735679872 +0000 UTC m=+680.615072359" Nov 04 05:39:06.864892 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:06.864819 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k" podStartSLOduration=1.864801295 podStartE2EDuration="1.864801295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:39:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:39:06.862449752 +0000 UTC m=+680.741842225" watchObservedRunningTime="2026-11-04 05:39:06.864801295 +0000 UTC m=+680.744193769" Nov 04 05:39:07.425116 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:07.425080 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" event={"ID":"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15","Type":"ContainerStarted","Data":"a979edbfe06ae280a2f2250734a0f9fcb5faef085aaf00aa6cd8d552f84dd384"} Nov 04 05:39:08.256193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:08.256148 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:39:08.395420 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:08.395382 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:39:08.396483 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:08.396456 5258 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Nov 04 05:39:08.396689 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:08.396671 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" podUID="6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Nov 04 05:39:09.396583 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:09.396545 5258 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Nov 04 05:39:09.397003 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:09.396611 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" podUID="6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Nov 04 05:39:10.396537 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:10.396503 5258 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Nov 04 05:39:10.396731 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:10.396551 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" podUID="6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Nov 04 05:39:10.903059 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:10.900061 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2"] Nov 04 05:39:10.920135 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:10.915026 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2" Nov 04 05:39:10.957110 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:10.950465 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2"] Nov 04 05:39:10.995353 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:10.995326 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-283815\"" Nov 04 05:39:10.995846 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:10.995825 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w5-283815-dockercfg-tc7jn\"" Nov 04 05:39:11.032715 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:11.032685 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63d9aa0a-c01b-46e5-b029-278cbeb206d6-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2\" (UID: \"63d9aa0a-c01b-46e5-b029-278cbeb206d6\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2" Nov 04 05:39:11.032847 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:11.032832 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wnfw\" (UniqueName: \"kubernetes.io/projected/63d9aa0a-c01b-46e5-b029-278cbeb206d6-kube-api-access-5wnfw\") pod \"payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2\" (UID: \"63d9aa0a-c01b-46e5-b029-278cbeb206d6\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2" Nov 04 05:39:11.150058 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:11.137665 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wnfw\" (UniqueName: \"kubernetes.io/projected/63d9aa0a-c01b-46e5-b029-278cbeb206d6-kube-api-access-5wnfw\") pod \"payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2\" (UID: \"63d9aa0a-c01b-46e5-b029-278cbeb206d6\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2" Nov 04 05:39:11.150058 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:11.137821 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63d9aa0a-c01b-46e5-b029-278cbeb206d6-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2\" (UID: \"63d9aa0a-c01b-46e5-b029-278cbeb206d6\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2" Nov 04 05:39:11.150058 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:11.138819 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63d9aa0a-c01b-46e5-b029-278cbeb206d6-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2\" (UID: \"63d9aa0a-c01b-46e5-b029-278cbeb206d6\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2" Nov 04 05:39:11.206242 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:11.206166 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wnfw\" (UniqueName: \"kubernetes.io/projected/63d9aa0a-c01b-46e5-b029-278cbeb206d6-kube-api-access-5wnfw\") pod \"payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2\" (UID: \"63d9aa0a-c01b-46e5-b029-278cbeb206d6\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2" Nov 04 05:39:11.230902 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:11.230382 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2" Nov 04 05:39:11.396801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:11.396360 5258 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Nov 04 05:39:11.396801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:11.396421 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" podUID="6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Nov 04 05:39:11.714871 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:11.714830 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2"] Nov 04 05:39:11.718701 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:39:11.718664 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63d9aa0a_c01b_46e5_b029_278cbeb206d6.slice/crio-e44b4c81bd5da65f700a31f1dbde87944f3f9a110e405f339cd238f9c73f4c20 WatchSource:0}: Error finding container e44b4c81bd5da65f700a31f1dbde87944f3f9a110e405f339cd238f9c73f4c20: Status 404 returned error can't find the container with id e44b4c81bd5da65f700a31f1dbde87944f3f9a110e405f339cd238f9c73f4c20 Nov 04 05:39:12.396080 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:12.395515 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt" Nov 04 05:39:12.396602 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:12.396306 5258 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Nov 04 05:39:12.396602 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:12.396348 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" podUID="6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Nov 04 05:39:12.396602 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:12.396576 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn" Nov 04 05:39:12.507966 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:12.507935 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2" event={"ID":"63d9aa0a-c01b-46e5-b029-278cbeb206d6","Type":"ContainerStarted","Data":"3b3a48ec342a44582544378c9830a0af3121aec789e6be19bbc94eaa5059f991"} Nov 04 05:39:12.508365 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:12.507983 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2" event={"ID":"63d9aa0a-c01b-46e5-b029-278cbeb206d6","Type":"ContainerStarted","Data":"e44b4c81bd5da65f700a31f1dbde87944f3f9a110e405f339cd238f9c73f4c20"} Nov 04 05:39:12.508365 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:12.508020 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2" Nov 04 05:39:12.796154 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:12.796043 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2" podStartSLOduration=2.796016152 podStartE2EDuration="2.796016152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:39:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:39:12.793744926 +0000 UTC m=+686.673137400" watchObservedRunningTime="2026-11-04 05:39:12.796016152 +0000 UTC m=+686.675408624" Nov 04 05:39:13.400887 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:13.400803 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:39:13.402260 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:13.402234 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:39:16.541181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:16.541094 5258 generic.go:356] "Generic (PLEG): container finished" podID="cf234545-c654-43c2-b9ee-a023302087f8" containerID="f190f45506b3f8f2b972d0bdfaea00beee8b19df0398064344cda34e657d346a" exitCode=0 Nov 04 05:39:16.541181 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:16.541158 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" event={"ID":"cf234545-c654-43c2-b9ee-a023302087f8","Type":"ContainerDied","Data":"f190f45506b3f8f2b972d0bdfaea00beee8b19df0398064344cda34e657d346a"} Nov 04 05:39:16.541626 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:16.541530 5258 scope.go:122] "RemoveContainer" containerID="f190f45506b3f8f2b972d0bdfaea00beee8b19df0398064344cda34e657d346a" Nov 04 05:39:16.542902 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:16.542877 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:39:16.542963 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:16.542940 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:39:16.543005 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:16.542970 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:39:17.433267 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:17.431412 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k" Nov 04 05:39:17.433267 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:17.432262 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf" Nov 04 05:39:17.566285 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:17.566250 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" event={"ID":"cf234545-c654-43c2-b9ee-a023302087f8","Type":"ContainerStarted","Data":"d4c7d22652894575687d4e36610c2b574338c31ccd5bc7c23a4c308f4c38ebb7"} Nov 04 05:39:17.627358 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:17.627328 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:39:17.765119 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:17.763418 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:39:17.765119 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:17.763529 5258 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Nov 04 05:39:17.765119 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:17.763571 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" podUID="cf234545-c654-43c2-b9ee-a023302087f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Nov 04 05:39:18.763519 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:18.763482 5258 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Nov 04 05:39:18.763940 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:18.763566 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" podUID="cf234545-c654-43c2-b9ee-a023302087f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Nov 04 05:39:19.762874 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:19.762834 5258 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Nov 04 05:39:19.763079 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:19.762892 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" podUID="cf234545-c654-43c2-b9ee-a023302087f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Nov 04 05:39:20.763526 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:20.763487 5258 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Nov 04 05:39:20.763918 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:20.763544 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" podUID="cf234545-c654-43c2-b9ee-a023302087f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Nov 04 05:39:21.763247 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:21.763209 5258 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Nov 04 05:39:21.763436 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:21.763260 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" podUID="cf234545-c654-43c2-b9ee-a023302087f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Nov 04 05:39:22.782484 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:39:22.782448 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:39:22.995365 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:22.995336 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:39:22.995787 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:22.995724 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:39:22.997930 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:22.997880 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:39:23.516677 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:23.516647 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2" Nov 04 05:39:27.543057 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:27.533496 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:39:37.706137 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:37.706103 5258 generic.go:356] "Generic (PLEG): container finished" podID="5cfab36f-9d70-4efa-96ed-a0db577b9939" containerID="1b0184779bba0f5c00fe4c33fc7bb8c68edb0ad56d17c138302fd4282a345935" exitCode=0 Nov 04 05:39:37.706615 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:37.706162 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" event={"ID":"5cfab36f-9d70-4efa-96ed-a0db577b9939","Type":"ContainerDied","Data":"1b0184779bba0f5c00fe4c33fc7bb8c68edb0ad56d17c138302fd4282a345935"} Nov 04 05:39:37.706615 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:37.706523 5258 scope.go:122] "RemoveContainer" containerID="1b0184779bba0f5c00fe4c33fc7bb8c68edb0ad56d17c138302fd4282a345935" Nov 04 05:39:37.707866 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:37.707832 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:39:37.707987 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:37.707919 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:39:37.707987 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:37.707967 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:39:38.713507 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:38.713471 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" event={"ID":"5cfab36f-9d70-4efa-96ed-a0db577b9939","Type":"ContainerStarted","Data":"33c053e899edc2496d9e1efe111467119cefe6f08d1e22dd4fe61837357e4c43"} Nov 04 05:39:38.997489 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:38.997401 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:39:38.998640 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:38.998611 5258 patch_prober.go:29] interesting pod/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Nov 04 05:39:38.998766 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:38.998678 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" podUID="5cfab36f-9d70-4efa-96ed-a0db577b9939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Nov 04 05:39:39.997547 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:39.997515 5258 patch_prober.go:29] interesting pod/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Nov 04 05:39:39.998026 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:39.998000 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" podUID="5cfab36f-9d70-4efa-96ed-a0db577b9939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Nov 04 05:39:40.260242 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:40.260141 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:39:40.998194 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:40.998155 5258 patch_prober.go:29] interesting pod/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Nov 04 05:39:40.998605 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:40.998222 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" podUID="5cfab36f-9d70-4efa-96ed-a0db577b9939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Nov 04 05:39:41.997327 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:41.997292 5258 patch_prober.go:29] interesting pod/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Nov 04 05:39:41.997536 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:41.997346 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" podUID="5cfab36f-9d70-4efa-96ed-a0db577b9939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Nov 04 05:39:42.996803 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:42.996766 5258 patch_prober.go:29] interesting pod/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Nov 04 05:39:42.997214 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:42.996836 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" podUID="5cfab36f-9d70-4efa-96ed-a0db577b9939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Nov 04 05:39:44.002435 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:44.002400 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:39:44.003435 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:44.003410 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:39:46.765934 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:46.765900 5258 generic.go:356] "Generic (PLEG): container finished" podID="86f487ad-d8ba-40ca-9f08-b4f560862516" containerID="8fc01b5d1790f5726d1ba0e0eece5ac03a9f09e1c4bbea055951536672444cee" exitCode=0 Nov 04 05:39:46.766563 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:46.765976 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" event={"ID":"86f487ad-d8ba-40ca-9f08-b4f560862516","Type":"ContainerDied","Data":"8fc01b5d1790f5726d1ba0e0eece5ac03a9f09e1c4bbea055951536672444cee"} Nov 04 05:39:46.766563 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:46.766480 5258 scope.go:122] "RemoveContainer" containerID="8fc01b5d1790f5726d1ba0e0eece5ac03a9f09e1c4bbea055951536672444cee" Nov 04 05:39:46.767834 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:46.767808 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:39:46.767916 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:46.767871 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:39:46.767916 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:46.767902 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:39:46.883525 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:46.883493 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n"] Nov 04 05:39:46.886427 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:46.886409 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:46.945582 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:46.945544 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n"] Nov 04 05:39:46.957767 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:46.957741 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-a84d48\"/\"kube-root-ca.crt\"" Nov 04 05:39:46.958931 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:46.958761 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-a84d48\"/\"simulator-w3-a84d48-kserve-self-signed-certs\"" Nov 04 05:39:46.959131 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:46.958829 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-a84d48\"/\"default-dockercfg-dvnsx\"" Nov 04 05:39:46.959195 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:46.958885 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-a84d48\"/\"openshift-service-ca.crt\"" Nov 04 05:39:46.979716 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:46.979682 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-model-cache\") pod \"simulator-w3-a84d48-kserve-67675f8597-z9g2n\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:46.979869 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:46.979841 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-dshm\") pod \"simulator-w3-a84d48-kserve-67675f8597-z9g2n\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:46.980012 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:46.979993 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13309444-7174-4fe6-9a47-d26d545a156f-tls-certs\") pod \"simulator-w3-a84d48-kserve-67675f8597-z9g2n\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:46.980118 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:46.980053 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49xgj\" (UniqueName: \"kubernetes.io/projected/13309444-7174-4fe6-9a47-d26d545a156f-kube-api-access-49xgj\") pod \"simulator-w3-a84d48-kserve-67675f8597-z9g2n\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:46.980118 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:46.980100 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-tmp-dir\") pod \"simulator-w3-a84d48-kserve-67675f8597-z9g2n\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:46.980240 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:46.980179 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-home\") pod \"simulator-w3-a84d48-kserve-67675f8597-z9g2n\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:47.081519 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:47.081439 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-dshm\") pod \"simulator-w3-a84d48-kserve-67675f8597-z9g2n\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:47.081665 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:47.081541 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13309444-7174-4fe6-9a47-d26d545a156f-tls-certs\") pod \"simulator-w3-a84d48-kserve-67675f8597-z9g2n\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:47.081665 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:47.081585 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49xgj\" (UniqueName: \"kubernetes.io/projected/13309444-7174-4fe6-9a47-d26d545a156f-kube-api-access-49xgj\") pod \"simulator-w3-a84d48-kserve-67675f8597-z9g2n\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:47.081665 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:47.081624 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-tmp-dir\") pod \"simulator-w3-a84d48-kserve-67675f8597-z9g2n\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:47.081665 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:47.081650 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-home\") pod \"simulator-w3-a84d48-kserve-67675f8597-z9g2n\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:47.081870 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:47.081706 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-model-cache\") pod \"simulator-w3-a84d48-kserve-67675f8597-z9g2n\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:47.082103 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:47.082074 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-home\") pod \"simulator-w3-a84d48-kserve-67675f8597-z9g2n\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:47.082205 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:47.082083 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-tmp-dir\") pod \"simulator-w3-a84d48-kserve-67675f8597-z9g2n\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:47.082205 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:47.082139 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-model-cache\") pod \"simulator-w3-a84d48-kserve-67675f8597-z9g2n\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:47.083789 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:47.083765 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-dshm\") pod \"simulator-w3-a84d48-kserve-67675f8597-z9g2n\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:47.084375 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:47.084358 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13309444-7174-4fe6-9a47-d26d545a156f-tls-certs\") pod \"simulator-w3-a84d48-kserve-67675f8597-z9g2n\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:47.148996 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:47.148965 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49xgj\" (UniqueName: \"kubernetes.io/projected/13309444-7174-4fe6-9a47-d26d545a156f-kube-api-access-49xgj\") pod \"simulator-w3-a84d48-kserve-67675f8597-z9g2n\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:47.197104 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:47.197072 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:47.215319 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:47.215282 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:39:47.616350 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:47.616314 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n"] Nov 04 05:39:47.619948 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:39:47.619918 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod13309444_7174_4fe6_9a47_d26d545a156f.slice/crio-b0d5b2a0bc6b942a5129d4d0a6933711c5c8266db8989bf447633e5ad69f4de7 WatchSource:0}: Error finding container b0d5b2a0bc6b942a5129d4d0a6933711c5c8266db8989bf447633e5ad69f4de7: Status 404 returned error can't find the container with id b0d5b2a0bc6b942a5129d4d0a6933711c5c8266db8989bf447633e5ad69f4de7 Nov 04 05:39:47.773389 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:47.773344 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" event={"ID":"86f487ad-d8ba-40ca-9f08-b4f560862516","Type":"ContainerStarted","Data":"49f1161a112cdd60e00c9717668533bc4bfe6d5059d1a52a243a65f9f4a3e4ee"} Nov 04 05:39:47.775636 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:47.775608 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" event={"ID":"13309444-7174-4fe6-9a47-d26d545a156f","Type":"ContainerStarted","Data":"cf1a202f861423c50c4c0bfb6549a8da57bb538d4bbc035d950a5cdc0f5f8207"} Nov 04 05:39:47.775768 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:47.775647 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" event={"ID":"13309444-7174-4fe6-9a47-d26d545a156f","Type":"ContainerStarted","Data":"b0d5b2a0bc6b942a5129d4d0a6933711c5c8266db8989bf447633e5ad69f4de7"} Nov 04 05:39:48.197965 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:48.197934 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:48.729438 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:48.729402 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:39:48.729625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:48.729603 5258 patch_prober.go:29] interesting pod/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Nov 04 05:39:48.729671 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:48.729649 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" podUID="86f487ad-d8ba-40ca-9f08-b4f560862516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Nov 04 05:39:49.729656 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:49.729619 5258 patch_prober.go:29] interesting pod/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Nov 04 05:39:49.730107 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:49.729671 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" podUID="86f487ad-d8ba-40ca-9f08-b4f560862516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Nov 04 05:39:50.729805 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:50.729767 5258 patch_prober.go:29] interesting pod/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Nov 04 05:39:50.730241 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:50.729819 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" podUID="86f487ad-d8ba-40ca-9f08-b4f560862516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Nov 04 05:39:51.729424 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:51.729392 5258 patch_prober.go:29] interesting pod/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Nov 04 05:39:51.729604 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:51.729442 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" podUID="86f487ad-d8ba-40ca-9f08-b4f560862516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Nov 04 05:39:52.729571 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:52.729530 5258 patch_prober.go:29] interesting pod/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Nov 04 05:39:52.730077 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:52.729590 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" podUID="86f487ad-d8ba-40ca-9f08-b4f560862516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Nov 04 05:39:52.781998 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:39:52.781963 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:39:53.736969 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:53.736935 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:39:53.737499 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:53.737320 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:39:53.738424 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:53.738402 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:39:53.983586 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:53.983488 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" podStartSLOduration=7.983473379 podStartE2EDuration="7.983473379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:39:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:39:48.052398599 +0000 UTC m=+721.931791111" watchObservedRunningTime="2026-11-04 05:39:53.983473379 +0000 UTC m=+727.862865852" Nov 04 05:39:57.197495 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:57.197453 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:57.210463 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:57.210436 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:39:57.842850 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:39:57.842817 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:40:02.205061 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:02.205012 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:40:14.768642 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:14.768610 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms"] Nov 04 05:40:14.772980 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:14.772960 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:14.890215 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:14.890180 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-901a00\"/\"openshift-service-ca.crt\"" Nov 04 05:40:14.890215 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:14.890188 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-901a00\"/\"kube-root-ca.crt\"" Nov 04 05:40:14.890215 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:14.890188 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-901a00\"/\"simulator-w4-901a00-kserve-self-signed-certs\"" Nov 04 05:40:14.890583 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:14.890251 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-901a00\"/\"default-dockercfg-8ts28\"" Nov 04 05:40:14.892439 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:14.892413 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms"] Nov 04 05:40:14.901290 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:14.901264 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-model-cache\") pod \"simulator-w4-901a00-kserve-658dfd7ddb-5wqms\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:14.901394 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:14.901307 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac5659ba-c807-44de-83f3-89ccd5eeac3b-tls-certs\") pod \"simulator-w4-901a00-kserve-658dfd7ddb-5wqms\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:14.901394 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:14.901333 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-tmp-dir\") pod \"simulator-w4-901a00-kserve-658dfd7ddb-5wqms\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:14.901475 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:14.901462 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-dshm\") pod \"simulator-w4-901a00-kserve-658dfd7ddb-5wqms\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:14.901510 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:14.901489 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-home\") pod \"simulator-w4-901a00-kserve-658dfd7ddb-5wqms\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:14.901543 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:14.901524 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl42r\" (UniqueName: \"kubernetes.io/projected/ac5659ba-c807-44de-83f3-89ccd5eeac3b-kube-api-access-hl42r\") pod \"simulator-w4-901a00-kserve-658dfd7ddb-5wqms\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:15.002333 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:15.002293 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl42r\" (UniqueName: \"kubernetes.io/projected/ac5659ba-c807-44de-83f3-89ccd5eeac3b-kube-api-access-hl42r\") pod \"simulator-w4-901a00-kserve-658dfd7ddb-5wqms\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:15.002522 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:15.002361 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-model-cache\") pod \"simulator-w4-901a00-kserve-658dfd7ddb-5wqms\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:15.002522 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:15.002395 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac5659ba-c807-44de-83f3-89ccd5eeac3b-tls-certs\") pod \"simulator-w4-901a00-kserve-658dfd7ddb-5wqms\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:15.002522 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:15.002418 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-tmp-dir\") pod \"simulator-w4-901a00-kserve-658dfd7ddb-5wqms\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:15.002522 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:15.002476 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-dshm\") pod \"simulator-w4-901a00-kserve-658dfd7ddb-5wqms\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:15.002522 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:15.002498 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-home\") pod \"simulator-w4-901a00-kserve-658dfd7ddb-5wqms\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:15.002821 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:15.002800 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-model-cache\") pod \"simulator-w4-901a00-kserve-658dfd7ddb-5wqms\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:15.002884 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:15.002828 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-tmp-dir\") pod \"simulator-w4-901a00-kserve-658dfd7ddb-5wqms\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:15.002949 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:15.002930 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-home\") pod \"simulator-w4-901a00-kserve-658dfd7ddb-5wqms\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:15.004793 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:15.004765 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-dshm\") pod \"simulator-w4-901a00-kserve-658dfd7ddb-5wqms\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:15.005074 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:15.005055 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac5659ba-c807-44de-83f3-89ccd5eeac3b-tls-certs\") pod \"simulator-w4-901a00-kserve-658dfd7ddb-5wqms\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:15.069975 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:15.069893 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl42r\" (UniqueName: \"kubernetes.io/projected/ac5659ba-c807-44de-83f3-89ccd5eeac3b-kube-api-access-hl42r\") pod \"simulator-w4-901a00-kserve-658dfd7ddb-5wqms\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:15.085747 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:15.085722 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:15.474235 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:15.474195 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms"] Nov 04 05:40:15.477946 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:40:15.477916 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac5659ba_c807_44de_83f3_89ccd5eeac3b.slice/crio-76582ffc158a56e0140731d72219a75a217d739e486f1efb9bb6b516617d577e WatchSource:0}: Error finding container 76582ffc158a56e0140731d72219a75a217d739e486f1efb9bb6b516617d577e: Status 404 returned error can't find the container with id 76582ffc158a56e0140731d72219a75a217d739e486f1efb9bb6b516617d577e Nov 04 05:40:15.937626 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:15.937584 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" event={"ID":"ac5659ba-c807-44de-83f3-89ccd5eeac3b","Type":"ContainerStarted","Data":"92cd0ad91e3f3e233fb34ae1c79f2add4d3efb0ed9019ef85abdd186d0796b46"} Nov 04 05:40:15.938139 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:15.937634 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" event={"ID":"ac5659ba-c807-44de-83f3-89ccd5eeac3b","Type":"ContainerStarted","Data":"76582ffc158a56e0140731d72219a75a217d739e486f1efb9bb6b516617d577e"} Nov 04 05:40:16.067803 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:16.067750 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" podStartSLOduration=2.067735709 podStartE2EDuration="2.067735709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:40:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:40:16.066710797 +0000 UTC m=+749.946103269" watchObservedRunningTime="2026-11-04 05:40:16.067735709 +0000 UTC m=+749.947128182" Nov 04 05:40:16.086062 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:16.086020 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:22.780455 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:40:22.780412 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:40:24.726743 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:24.726704 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:40:25.086979 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:25.086887 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:25.099968 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:25.099933 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:26.024053 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:26.024008 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:40:29.402669 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.402635 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9"] Nov 04 05:40:29.406491 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.406475 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:29.465310 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.465274 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-098178\"/\"openshift-service-ca.crt\"" Nov 04 05:40:29.521568 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.521531 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-098178\"/\"kube-root-ca.crt\"" Nov 04 05:40:29.521732 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.521580 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-098178\"/\"simulator-w1-098178-kserve-self-signed-certs\"" Nov 04 05:40:29.521732 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.521623 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-098178\"/\"default-dockercfg-cqcz7\"" Nov 04 05:40:29.523877 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.523844 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9"] Nov 04 05:40:29.564841 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.564788 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-dshm\") pod \"simulator-w1-098178-kserve-78f896649-qprd9\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:29.565071 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.564863 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bltck\" (UniqueName: \"kubernetes.io/projected/07d861a6-b569-4144-83a7-6376c2ab6514-kube-api-access-bltck\") pod \"simulator-w1-098178-kserve-78f896649-qprd9\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:29.565138 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.565059 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07d861a6-b569-4144-83a7-6376c2ab6514-tls-certs\") pod \"simulator-w1-098178-kserve-78f896649-qprd9\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:29.565233 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.565201 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-model-cache\") pod \"simulator-w1-098178-kserve-78f896649-qprd9\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:29.565310 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.565295 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-tmp-dir\") pod \"simulator-w1-098178-kserve-78f896649-qprd9\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:29.565371 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.565327 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-home\") pod \"simulator-w1-098178-kserve-78f896649-qprd9\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:29.666646 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.666565 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-dshm\") pod \"simulator-w1-098178-kserve-78f896649-qprd9\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:29.666646 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.666609 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bltck\" (UniqueName: \"kubernetes.io/projected/07d861a6-b569-4144-83a7-6376c2ab6514-kube-api-access-bltck\") pod \"simulator-w1-098178-kserve-78f896649-qprd9\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:29.666843 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.666660 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07d861a6-b569-4144-83a7-6376c2ab6514-tls-certs\") pod \"simulator-w1-098178-kserve-78f896649-qprd9\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:29.666843 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.666729 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-model-cache\") pod \"simulator-w1-098178-kserve-78f896649-qprd9\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:29.666843 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.666755 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-tmp-dir\") pod \"simulator-w1-098178-kserve-78f896649-qprd9\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:29.666843 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.666776 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-home\") pod \"simulator-w1-098178-kserve-78f896649-qprd9\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:29.667131 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.667104 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-model-cache\") pod \"simulator-w1-098178-kserve-78f896649-qprd9\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:29.667215 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.667159 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-home\") pod \"simulator-w1-098178-kserve-78f896649-qprd9\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:29.667215 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.667176 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-tmp-dir\") pod \"simulator-w1-098178-kserve-78f896649-qprd9\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:29.668928 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.668901 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-dshm\") pod \"simulator-w1-098178-kserve-78f896649-qprd9\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:29.669118 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.669101 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07d861a6-b569-4144-83a7-6376c2ab6514-tls-certs\") pod \"simulator-w1-098178-kserve-78f896649-qprd9\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:29.734235 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:29.734198 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bltck\" (UniqueName: \"kubernetes.io/projected/07d861a6-b569-4144-83a7-6376c2ab6514-kube-api-access-bltck\") pod \"simulator-w1-098178-kserve-78f896649-qprd9\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:30.019231 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:30.019137 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:30.408386 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:30.408354 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9"] Nov 04 05:40:30.411800 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:40:30.411776 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07d861a6_b569_4144_83a7_6376c2ab6514.slice/crio-8b3dcb1f835f42d508563559885ddbfa2ef9610990db262fa9d33e0077861e45 WatchSource:0}: Error finding container 8b3dcb1f835f42d508563559885ddbfa2ef9610990db262fa9d33e0077861e45: Status 404 returned error can't find the container with id 8b3dcb1f835f42d508563559885ddbfa2ef9610990db262fa9d33e0077861e45 Nov 04 05:40:31.042179 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:31.042138 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" event={"ID":"07d861a6-b569-4144-83a7-6376c2ab6514","Type":"ContainerStarted","Data":"f56dbc3a028421868bdf329053cd5c04d3bcd0ac0dfa3e0cd4d75f35293f43ae"} Nov 04 05:40:31.042179 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:31.042184 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" event={"ID":"07d861a6-b569-4144-83a7-6376c2ab6514","Type":"ContainerStarted","Data":"8b3dcb1f835f42d508563559885ddbfa2ef9610990db262fa9d33e0077861e45"} Nov 04 05:40:31.173243 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:31.173192 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" podStartSLOduration=2.173177587 podStartE2EDuration="2.173177587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:40:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:40:31.171333006 +0000 UTC m=+765.050725478" watchObservedRunningTime="2026-11-04 05:40:31.173177587 +0000 UTC m=+765.052570060" Nov 04 05:40:32.019673 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:32.019641 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:40.019595 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:40.019562 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:40.032706 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:40.032681 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:40.107225 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:40.107198 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:40:44.350100 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:44.350059 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:40:49.049933 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.049899 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8"] Nov 04 05:40:49.055133 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.055111 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:40:49.139624 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.139589 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-90dc8d\"/\"openshift-service-ca.crt\"" Nov 04 05:40:49.139819 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.139783 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-90dc8d\"/\"simulator-w0-90dc8d-kserve-self-signed-certs\"" Nov 04 05:40:49.142470 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.142446 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8"] Nov 04 05:40:49.172075 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.172021 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-90dc8d\"/\"kube-root-ca.crt\"" Nov 04 05:40:49.172075 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.172065 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-90dc8d\"/\"default-dockercfg-l4t9d\"" Nov 04 05:40:49.174574 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.174549 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-dshm\") pod \"simulator-w0-90dc8d-kserve-59896b544c-9mrw8\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:40:49.174646 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.174583 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-model-cache\") pod \"simulator-w0-90dc8d-kserve-59896b544c-9mrw8\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:40:49.174646 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.174605 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-tls-certs\") pod \"simulator-w0-90dc8d-kserve-59896b544c-9mrw8\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:40:49.174804 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.174785 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-home\") pod \"simulator-w0-90dc8d-kserve-59896b544c-9mrw8\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:40:49.174868 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.174851 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2b7f\" (UniqueName: \"kubernetes.io/projected/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-kube-api-access-t2b7f\") pod \"simulator-w0-90dc8d-kserve-59896b544c-9mrw8\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:40:49.174929 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.174910 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-tmp-dir\") pod \"simulator-w0-90dc8d-kserve-59896b544c-9mrw8\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:40:49.276313 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.276279 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-model-cache\") pod \"simulator-w0-90dc8d-kserve-59896b544c-9mrw8\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:40:49.276313 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.276318 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-tls-certs\") pod \"simulator-w0-90dc8d-kserve-59896b544c-9mrw8\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:40:49.276555 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.276369 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-home\") pod \"simulator-w0-90dc8d-kserve-59896b544c-9mrw8\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:40:49.276555 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.276403 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2b7f\" (UniqueName: \"kubernetes.io/projected/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-kube-api-access-t2b7f\") pod \"simulator-w0-90dc8d-kserve-59896b544c-9mrw8\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:40:49.276555 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.276486 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-tmp-dir\") pod \"simulator-w0-90dc8d-kserve-59896b544c-9mrw8\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:40:49.276740 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.276635 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-dshm\") pod \"simulator-w0-90dc8d-kserve-59896b544c-9mrw8\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:40:49.276740 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.276716 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-model-cache\") pod \"simulator-w0-90dc8d-kserve-59896b544c-9mrw8\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:40:49.276861 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.276774 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-home\") pod \"simulator-w0-90dc8d-kserve-59896b544c-9mrw8\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:40:49.276920 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.276850 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-tmp-dir\") pod \"simulator-w0-90dc8d-kserve-59896b544c-9mrw8\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:40:49.278750 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.278716 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-dshm\") pod \"simulator-w0-90dc8d-kserve-59896b544c-9mrw8\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:40:49.278861 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.278853 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-tls-certs\") pod \"simulator-w0-90dc8d-kserve-59896b544c-9mrw8\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:40:49.346482 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.346407 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2b7f\" (UniqueName: \"kubernetes.io/projected/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-kube-api-access-t2b7f\") pod \"simulator-w0-90dc8d-kserve-59896b544c-9mrw8\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:40:49.367297 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.367265 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:40:49.756749 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:49.756712 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8"] Nov 04 05:40:49.760264 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:40:49.760238 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7328ba4a_fae2_48c7_8d74_6dfbb0ec37e3.slice/crio-53ccb6f1726d4a43add44ab39cea7946765c635943fce44c3a051624000ebf79 WatchSource:0}: Error finding container 53ccb6f1726d4a43add44ab39cea7946765c635943fce44c3a051624000ebf79: Status 404 returned error can't find the container with id 53ccb6f1726d4a43add44ab39cea7946765c635943fce44c3a051624000ebf79 Nov 04 05:40:50.151350 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:50.151317 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" event={"ID":"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3","Type":"ContainerStarted","Data":"586637b8368756ae6c11bb8936aa6eab2b51e1bd5f435d2232c422c63ab234d5"} Nov 04 05:40:50.151350 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:50.151357 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" event={"ID":"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3","Type":"ContainerStarted","Data":"53ccb6f1726d4a43add44ab39cea7946765c635943fce44c3a051624000ebf79"} Nov 04 05:40:50.282587 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:50.282533 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" podStartSLOduration=1.282519099 podStartE2EDuration="1.282519099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:40:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:40:50.280414346 +0000 UTC m=+784.159806819" watchObservedRunningTime="2026-11-04 05:40:50.282519099 +0000 UTC m=+784.161911625" Nov 04 05:40:50.367538 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:50.367501 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:40:52.779980 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:40:52.779945 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:40:59.368485 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:59.368448 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:40:59.381176 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:40:59.381146 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:41:00.214723 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:41:00.214690 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:41:22.781296 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:41:22.781249 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:41:33.238922 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:41:33.238843 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:41:52.781194 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:41:52.781152 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:42:02.557671 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:02.557636 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:42:22.373683 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.373642 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh"] Nov 04 05:42:22.377579 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.377559 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:22.436584 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.436554 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-90dc8d\"/\"premium-w0-90dc8d-kserve-self-signed-certs\"" Nov 04 05:42:22.495007 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.494970 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh"] Nov 04 05:42:22.495390 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.495351 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-tmp-dir\") pod \"premium-w0-90dc8d-kserve-79874fbf4b-jtgkh\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:22.495440 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.495409 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfs6h\" (UniqueName: \"kubernetes.io/projected/9e98783f-2552-48ab-b17a-8d1d0d214950-kube-api-access-zfs6h\") pod \"premium-w0-90dc8d-kserve-79874fbf4b-jtgkh\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:22.495440 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.495429 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-home\") pod \"premium-w0-90dc8d-kserve-79874fbf4b-jtgkh\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:22.495533 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.495521 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e98783f-2552-48ab-b17a-8d1d0d214950-tls-certs\") pod \"premium-w0-90dc8d-kserve-79874fbf4b-jtgkh\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:22.495584 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.495574 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-dshm\") pod \"premium-w0-90dc8d-kserve-79874fbf4b-jtgkh\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:22.495620 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.495591 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-model-cache\") pod \"premium-w0-90dc8d-kserve-79874fbf4b-jtgkh\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:22.597210 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.597171 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-dshm\") pod \"premium-w0-90dc8d-kserve-79874fbf4b-jtgkh\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:22.597210 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.597207 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-model-cache\") pod \"premium-w0-90dc8d-kserve-79874fbf4b-jtgkh\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:22.597443 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.597272 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-tmp-dir\") pod \"premium-w0-90dc8d-kserve-79874fbf4b-jtgkh\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:22.597443 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.597301 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfs6h\" (UniqueName: \"kubernetes.io/projected/9e98783f-2552-48ab-b17a-8d1d0d214950-kube-api-access-zfs6h\") pod \"premium-w0-90dc8d-kserve-79874fbf4b-jtgkh\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:22.597443 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.597320 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-home\") pod \"premium-w0-90dc8d-kserve-79874fbf4b-jtgkh\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:22.597443 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.597382 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e98783f-2552-48ab-b17a-8d1d0d214950-tls-certs\") pod \"premium-w0-90dc8d-kserve-79874fbf4b-jtgkh\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:22.597748 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.597719 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-tmp-dir\") pod \"premium-w0-90dc8d-kserve-79874fbf4b-jtgkh\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:22.597748 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.597741 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-model-cache\") pod \"premium-w0-90dc8d-kserve-79874fbf4b-jtgkh\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:22.597937 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.597792 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-home\") pod \"premium-w0-90dc8d-kserve-79874fbf4b-jtgkh\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:22.599406 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.599383 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-dshm\") pod \"premium-w0-90dc8d-kserve-79874fbf4b-jtgkh\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:22.599723 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.599707 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e98783f-2552-48ab-b17a-8d1d0d214950-tls-certs\") pod \"premium-w0-90dc8d-kserve-79874fbf4b-jtgkh\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:22.665096 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.664954 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfs6h\" (UniqueName: \"kubernetes.io/projected/9e98783f-2552-48ab-b17a-8d1d0d214950-kube-api-access-zfs6h\") pod \"premium-w0-90dc8d-kserve-79874fbf4b-jtgkh\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:22.689864 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:22.689833 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:22.810175 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:42:22.810141 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:42:23.077624 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:23.077597 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh"] Nov 04 05:42:23.080492 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:42:23.080463 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e98783f_2552_48ab_b17a_8d1d0d214950.slice/crio-739a2f77f4a6d40e90a99145c212561d05f85a9ff7a85dbd2919a025c99bb783 WatchSource:0}: Error finding container 739a2f77f4a6d40e90a99145c212561d05f85a9ff7a85dbd2919a025c99bb783: Status 404 returned error can't find the container with id 739a2f77f4a6d40e90a99145c212561d05f85a9ff7a85dbd2919a025c99bb783 Nov 04 05:42:23.657969 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:23.657931 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" event={"ID":"9e98783f-2552-48ab-b17a-8d1d0d214950","Type":"ContainerStarted","Data":"bf512bc7d159d174c97eb0f7e799dec08c9f8b1e7b4d1f14832ab37abf3c0842"} Nov 04 05:42:23.658362 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:23.657981 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" event={"ID":"9e98783f-2552-48ab-b17a-8d1d0d214950","Type":"ContainerStarted","Data":"739a2f77f4a6d40e90a99145c212561d05f85a9ff7a85dbd2919a025c99bb783"} Nov 04 05:42:23.690461 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:23.690436 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:23.790686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:23.790627 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" podStartSLOduration=1.790610343 podStartE2EDuration="1.790610343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:42:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:42:23.788439336 +0000 UTC m=+877.667831808" watchObservedRunningTime="2026-11-04 05:42:23.790610343 +0000 UTC m=+877.670002815" Nov 04 05:42:32.690625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:32.690585 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:32.704061 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:32.704023 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:32.722852 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:32.722816 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:42:35.129957 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:35.129922 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:42:52.781511 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:42:52.781471 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:42:54.044418 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.044345 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb"] Nov 04 05:42:54.048456 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.048438 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.112182 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.112152 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-90dc8d-openshift-default-dockercfg-8rgkv\"" Nov 04 05:42:54.142979 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.142940 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-token\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.143166 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.142991 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-data\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.143166 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.143149 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-envoy\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.143240 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.143182 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-credential-socket\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.143275 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.143263 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-podinfo\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.143308 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.143288 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-workload-certs\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.143347 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.143305 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcnph\" (UniqueName: \"kubernetes.io/projected/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-kube-api-access-hcnph\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.143393 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.143381 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-workload-socket\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.143435 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.143424 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istiod-ca-cert\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.170388 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.170351 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb"] Nov 04 05:42:54.244426 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.244396 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-workload-certs\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.244426 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.244428 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcnph\" (UniqueName: \"kubernetes.io/projected/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-kube-api-access-hcnph\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.244675 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.244570 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-workload-socket\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.244675 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.244630 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istiod-ca-cert\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.244781 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.244728 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-token\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.244847 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.244781 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-data\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.244847 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.244818 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-workload-certs\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.244957 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.244896 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-workload-socket\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.244957 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.244910 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-envoy\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.245104 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.244954 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-credential-socket\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.245104 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.245015 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-podinfo\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.245217 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.245190 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-data\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.245317 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.245288 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-credential-socket\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.245428 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.245403 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istiod-ca-cert\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.247018 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.246998 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-envoy\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.247394 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.247377 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-podinfo\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.311690 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.311618 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcnph\" (UniqueName: \"kubernetes.io/projected/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-kube-api-access-hcnph\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.311690 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.311620 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-token\") pod \"e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.361822 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.361784 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:54.812119 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.811947 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb"] Nov 04 05:42:54.814896 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:42:54.814867 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e00e063_7fc1_4104_b3b5_4f41f8926b5b.slice/crio-502e7acfc3ffe3052b3060a2532618fcf1c7e0d9723ce481bb80812ce6217cf4 WatchSource:0}: Error finding container 502e7acfc3ffe3052b3060a2532618fcf1c7e0d9723ce481bb80812ce6217cf4: Status 404 returned error can't find the container with id 502e7acfc3ffe3052b3060a2532618fcf1c7e0d9723ce481bb80812ce6217cf4 Nov 04 05:42:54.817243 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.817214 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:42:54.817304 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.817277 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:42:54.817339 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.817303 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:42:54.827281 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:54.827259 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" event={"ID":"8e00e063-7fc1-4104-b3b5-4f41f8926b5b","Type":"ContainerStarted","Data":"502e7acfc3ffe3052b3060a2532618fcf1c7e0d9723ce481bb80812ce6217cf4"} Nov 04 05:42:55.858607 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:55.858570 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" event={"ID":"8e00e063-7fc1-4104-b3b5-4f41f8926b5b","Type":"ContainerStarted","Data":"0b13eee03f12889e72563f790ab4add2f48b52e67e9aa329a79fe3e0a0b55e2e"} Nov 04 05:42:56.058120 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:56.057531 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" podStartSLOduration=3.0575087 podStartE2EDuration="3.0575087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:42:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:42:56.054514468 +0000 UTC m=+909.933906949" watchObservedRunningTime="2026-11-04 05:42:56.0575087 +0000 UTC m=+909.936901176" Nov 04 05:42:56.366517 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:56.362511 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:42:56.368160 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:56.368117 5258 patch_prober.go:29] interesting pod/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Nov 04 05:42:56.369006 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:56.368967 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" podUID="8e00e063-7fc1-4104-b3b5-4f41f8926b5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Nov 04 05:42:57.362919 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:57.362882 5258 patch_prober.go:29] interesting pod/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Nov 04 05:42:57.363346 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:57.362969 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" podUID="8e00e063-7fc1-4104-b3b5-4f41f8926b5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Nov 04 05:42:58.362654 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:58.362617 5258 patch_prober.go:29] interesting pod/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Nov 04 05:42:58.362872 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:58.362696 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" podUID="8e00e063-7fc1-4104-b3b5-4f41f8926b5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Nov 04 05:42:59.363214 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:59.363174 5258 patch_prober.go:29] interesting pod/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Nov 04 05:42:59.363627 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:42:59.363232 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" podUID="8e00e063-7fc1-4104-b3b5-4f41f8926b5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Nov 04 05:43:00.362443 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:43:00.362406 5258 patch_prober.go:29] interesting pod/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body= Nov 04 05:43:00.362620 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:43:00.362465 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" podUID="8e00e063-7fc1-4104-b3b5-4f41f8926b5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" Nov 04 05:43:01.578605 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:43:01.578567 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:43:01.579190 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:43:01.578967 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:43:01.579917 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:43:01.579898 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:43:18.161403 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:43:18.161364 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:43:22.780277 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:43:22.780239 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:43:52.780182 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:43:52.780142 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:44:01.475982 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:01.475941 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:44:11.645657 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:11.645610 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl"] Nov 04 05:44:11.653376 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:11.653348 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl" Nov 04 05:44:11.711633 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:11.711602 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl"] Nov 04 05:44:11.753950 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:11.753918 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1f42dd6a-1809-4cb1-af96-f23fa8cb693c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-db136c12-pk5gl\" (UID: \"1f42dd6a-1809-4cb1-af96-f23fa8cb693c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl" Nov 04 05:44:11.754144 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:11.754069 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwvh4\" (UniqueName: \"kubernetes.io/projected/1f42dd6a-1809-4cb1-af96-f23fa8cb693c-kube-api-access-wwvh4\") pod \"maas-api-revoke-sub-db136c12-pk5gl\" (UID: \"1f42dd6a-1809-4cb1-af96-f23fa8cb693c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl" Nov 04 05:44:11.754219 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:11.754179 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1f42dd6a-1809-4cb1-af96-f23fa8cb693c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-db136c12-pk5gl\" (UID: \"1f42dd6a-1809-4cb1-af96-f23fa8cb693c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl" Nov 04 05:44:11.764711 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:11.764685 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 05:44:11.764886 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:11.764862 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 05:44:11.780664 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:11.780641 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dnddh\"" Nov 04 05:44:11.780777 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:11.780671 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Nov 04 05:44:11.854882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:11.854853 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1f42dd6a-1809-4cb1-af96-f23fa8cb693c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-db136c12-pk5gl\" (UID: \"1f42dd6a-1809-4cb1-af96-f23fa8cb693c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl" Nov 04 05:44:11.855068 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:11.854915 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwvh4\" (UniqueName: \"kubernetes.io/projected/1f42dd6a-1809-4cb1-af96-f23fa8cb693c-kube-api-access-wwvh4\") pod \"maas-api-revoke-sub-db136c12-pk5gl\" (UID: \"1f42dd6a-1809-4cb1-af96-f23fa8cb693c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl" Nov 04 05:44:11.855068 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:11.854963 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1f42dd6a-1809-4cb1-af96-f23fa8cb693c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-db136c12-pk5gl\" (UID: \"1f42dd6a-1809-4cb1-af96-f23fa8cb693c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl" Nov 04 05:44:11.855556 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:11.855533 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1f42dd6a-1809-4cb1-af96-f23fa8cb693c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-db136c12-pk5gl\" (UID: \"1f42dd6a-1809-4cb1-af96-f23fa8cb693c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl" Nov 04 05:44:11.857340 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:11.857316 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1f42dd6a-1809-4cb1-af96-f23fa8cb693c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-db136c12-pk5gl\" (UID: \"1f42dd6a-1809-4cb1-af96-f23fa8cb693c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl" Nov 04 05:44:11.922555 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:11.922488 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwvh4\" (UniqueName: \"kubernetes.io/projected/1f42dd6a-1809-4cb1-af96-f23fa8cb693c-kube-api-access-wwvh4\") pod \"maas-api-revoke-sub-db136c12-pk5gl\" (UID: \"1f42dd6a-1809-4cb1-af96-f23fa8cb693c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl" Nov 04 05:44:11.967643 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:11.967619 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl" Nov 04 05:44:12.380484 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:12.380409 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl"] Nov 04 05:44:12.383531 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:44:12.383505 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f42dd6a_1809_4cb1_af96_f23fa8cb693c.slice/crio-1ca4712ce8788aef07a9b3ab9205d9cbfcb9096814986f1ca8c0e04a403fc4d1 WatchSource:0}: Error finding container 1ca4712ce8788aef07a9b3ab9205d9cbfcb9096814986f1ca8c0e04a403fc4d1: Status 404 returned error can't find the container with id 1ca4712ce8788aef07a9b3ab9205d9cbfcb9096814986f1ca8c0e04a403fc4d1 Nov 04 05:44:12.385489 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:12.385471 5258 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 05:44:13.348380 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:13.348343 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl" event={"ID":"1f42dd6a-1809-4cb1-af96-f23fa8cb693c","Type":"ContainerStarted","Data":"1ca4712ce8788aef07a9b3ab9205d9cbfcb9096814986f1ca8c0e04a403fc4d1"} Nov 04 05:44:13.620954 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:13.620913 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr"] Nov 04 05:44:13.625952 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:13.625935 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr" Nov 04 05:44:13.688731 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:13.688694 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr"] Nov 04 05:44:13.776860 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:13.776824 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a0cad87e-348b-45d8-a9b9-cdc83731dbd3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2f464bb2-gpqlr\" (UID: \"a0cad87e-348b-45d8-a9b9-cdc83731dbd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr" Nov 04 05:44:13.777016 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:13.776879 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47zc7\" (UniqueName: \"kubernetes.io/projected/a0cad87e-348b-45d8-a9b9-cdc83731dbd3-kube-api-access-47zc7\") pod \"maas-api-revoke-sub-2f464bb2-gpqlr\" (UID: \"a0cad87e-348b-45d8-a9b9-cdc83731dbd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr" Nov 04 05:44:13.777107 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:13.777047 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a0cad87e-348b-45d8-a9b9-cdc83731dbd3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2f464bb2-gpqlr\" (UID: \"a0cad87e-348b-45d8-a9b9-cdc83731dbd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr" Nov 04 05:44:13.878406 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:13.878319 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a0cad87e-348b-45d8-a9b9-cdc83731dbd3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2f464bb2-gpqlr\" (UID: \"a0cad87e-348b-45d8-a9b9-cdc83731dbd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr" Nov 04 05:44:13.878406 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:13.878360 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47zc7\" (UniqueName: \"kubernetes.io/projected/a0cad87e-348b-45d8-a9b9-cdc83731dbd3-kube-api-access-47zc7\") pod \"maas-api-revoke-sub-2f464bb2-gpqlr\" (UID: \"a0cad87e-348b-45d8-a9b9-cdc83731dbd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr" Nov 04 05:44:13.878406 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:13.878404 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a0cad87e-348b-45d8-a9b9-cdc83731dbd3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2f464bb2-gpqlr\" (UID: \"a0cad87e-348b-45d8-a9b9-cdc83731dbd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr" Nov 04 05:44:13.878941 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:13.878920 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a0cad87e-348b-45d8-a9b9-cdc83731dbd3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2f464bb2-gpqlr\" (UID: \"a0cad87e-348b-45d8-a9b9-cdc83731dbd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr" Nov 04 05:44:13.880748 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:13.880724 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a0cad87e-348b-45d8-a9b9-cdc83731dbd3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2f464bb2-gpqlr\" (UID: \"a0cad87e-348b-45d8-a9b9-cdc83731dbd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr" Nov 04 05:44:13.946068 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:13.946007 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47zc7\" (UniqueName: \"kubernetes.io/projected/a0cad87e-348b-45d8-a9b9-cdc83731dbd3-kube-api-access-47zc7\") pod \"maas-api-revoke-sub-2f464bb2-gpqlr\" (UID: \"a0cad87e-348b-45d8-a9b9-cdc83731dbd3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr" Nov 04 05:44:14.240005 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:14.239926 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr" Nov 04 05:44:14.626774 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:14.626747 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr"] Nov 04 05:44:14.630001 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:44:14.629971 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0cad87e_348b_45d8_a9b9_cdc83731dbd3.slice/crio-92fc51986722465bb174e9555c3b1b970b10ae5cd962b2b58341b54336ce98b0 WatchSource:0}: Error finding container 92fc51986722465bb174e9555c3b1b970b10ae5cd962b2b58341b54336ce98b0: Status 404 returned error can't find the container with id 92fc51986722465bb174e9555c3b1b970b10ae5cd962b2b58341b54336ce98b0 Nov 04 05:44:15.365138 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:15.365099 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr" event={"ID":"a0cad87e-348b-45d8-a9b9-cdc83731dbd3","Type":"ContainerStarted","Data":"92fc51986722465bb174e9555c3b1b970b10ae5cd962b2b58341b54336ce98b0"} Nov 04 05:44:15.857136 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:15.857103 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8"] Nov 04 05:44:15.861239 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:15.861210 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8" Nov 04 05:44:15.923566 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:15.923520 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9"] Nov 04 05:44:15.980698 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:15.980669 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8"] Nov 04 05:44:16.006892 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:16.006858 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w42cz\" (UniqueName: \"kubernetes.io/projected/7ef5af96-976f-49c3-9d61-ca2cc1369bf2-kube-api-access-w42cz\") pod \"maas-api-revoke-sub-7b4ebc2d-w7xb8\" (UID: \"7ef5af96-976f-49c3-9d61-ca2cc1369bf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8" Nov 04 05:44:16.007096 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:16.006908 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ef5af96-976f-49c3-9d61-ca2cc1369bf2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7b4ebc2d-w7xb8\" (UID: \"7ef5af96-976f-49c3-9d61-ca2cc1369bf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8" Nov 04 05:44:16.007096 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:16.006957 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ef5af96-976f-49c3-9d61-ca2cc1369bf2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7b4ebc2d-w7xb8\" (UID: \"7ef5af96-976f-49c3-9d61-ca2cc1369bf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8" Nov 04 05:44:16.108288 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:16.108205 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w42cz\" (UniqueName: \"kubernetes.io/projected/7ef5af96-976f-49c3-9d61-ca2cc1369bf2-kube-api-access-w42cz\") pod \"maas-api-revoke-sub-7b4ebc2d-w7xb8\" (UID: \"7ef5af96-976f-49c3-9d61-ca2cc1369bf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8" Nov 04 05:44:16.108288 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:16.108260 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ef5af96-976f-49c3-9d61-ca2cc1369bf2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7b4ebc2d-w7xb8\" (UID: \"7ef5af96-976f-49c3-9d61-ca2cc1369bf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8" Nov 04 05:44:16.108535 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:16.108311 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ef5af96-976f-49c3-9d61-ca2cc1369bf2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7b4ebc2d-w7xb8\" (UID: \"7ef5af96-976f-49c3-9d61-ca2cc1369bf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8" Nov 04 05:44:16.109025 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:16.108995 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ef5af96-976f-49c3-9d61-ca2cc1369bf2-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-7b4ebc2d-w7xb8\" (UID: \"7ef5af96-976f-49c3-9d61-ca2cc1369bf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8" Nov 04 05:44:16.110989 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:16.110965 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ef5af96-976f-49c3-9d61-ca2cc1369bf2-maas-api-xks-ca\") pod \"maas-api-revoke-sub-7b4ebc2d-w7xb8\" (UID: \"7ef5af96-976f-49c3-9d61-ca2cc1369bf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8" Nov 04 05:44:16.176336 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:16.176298 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w42cz\" (UniqueName: \"kubernetes.io/projected/7ef5af96-976f-49c3-9d61-ca2cc1369bf2-kube-api-access-w42cz\") pod \"maas-api-revoke-sub-7b4ebc2d-w7xb8\" (UID: \"7ef5af96-976f-49c3-9d61-ca2cc1369bf2\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8" Nov 04 05:44:16.181374 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:16.181347 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8" Nov 04 05:44:16.584502 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:16.584472 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8"] Nov 04 05:44:16.587830 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:44:16.587802 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ef5af96_976f_49c3_9d61_ca2cc1369bf2.slice/crio-8d0c27d9a6038188aec4663da2a8e053a22dd409b7c17b7001e383a48412da84 WatchSource:0}: Error finding container 8d0c27d9a6038188aec4663da2a8e053a22dd409b7c17b7001e383a48412da84: Status 404 returned error can't find the container with id 8d0c27d9a6038188aec4663da2a8e053a22dd409b7c17b7001e383a48412da84 Nov 04 05:44:17.381912 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:17.381877 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8" event={"ID":"7ef5af96-976f-49c3-9d61-ca2cc1369bf2","Type":"ContainerStarted","Data":"8d0c27d9a6038188aec4663da2a8e053a22dd409b7c17b7001e383a48412da84"} Nov 04 05:44:17.384393 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:17.384357 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl" event={"ID":"1f42dd6a-1809-4cb1-af96-f23fa8cb693c","Type":"ContainerStarted","Data":"01fcc57b123c283fbc8a292e5c30170463f3d49f521974d3b62b1d918c7fc342"} Nov 04 05:44:17.516170 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:17.516116 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl" podStartSLOduration=2.093218347 podStartE2EDuration="6.516092767s" totalImagesPullingTime="4.42287442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:44:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:44:17.514237648 +0000 UTC m=+991.393630122" watchObservedRunningTime="2026-11-04 05:44:17.516092767 +0000 UTC m=+991.395485242" Nov 04 05:44:18.391743 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:18.391708 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr" event={"ID":"a0cad87e-348b-45d8-a9b9-cdc83731dbd3","Type":"ContainerStarted","Data":"9d021ce8eac1ccdb63fd79a37b95b7a5c7a3840c139a374a9cf5343934bd2e86"} Nov 04 05:44:18.393317 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:18.393285 5258 generic.go:356] "Generic (PLEG): container finished" podID="1f42dd6a-1809-4cb1-af96-f23fa8cb693c" containerID="01fcc57b123c283fbc8a292e5c30170463f3d49f521974d3b62b1d918c7fc342" exitCode=0 Nov 04 05:44:18.393425 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:18.393350 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl" event={"ID":"1f42dd6a-1809-4cb1-af96-f23fa8cb693c","Type":"ContainerDied","Data":"01fcc57b123c283fbc8a292e5c30170463f3d49f521974d3b62b1d918c7fc342"} Nov 04 05:44:18.522276 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:18.522217 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr" podStartSLOduration=2.51240907 podStartE2EDuration="5.522202894s" totalImagesPullingTime="3.009793824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:44:18.519437571 +0000 UTC m=+992.398830043" watchObservedRunningTime="2026-11-04 05:44:18.522202894 +0000 UTC m=+992.401595367" Nov 04 05:44:19.400902 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:19.400852 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8" event={"ID":"7ef5af96-976f-49c3-9d61-ca2cc1369bf2","Type":"ContainerStarted","Data":"5eb508184b37d1aac731b6d546fc4e622565352dd52864121d4f4a515e222614"} Nov 04 05:44:19.402447 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:19.402423 5258 generic.go:356] "Generic (PLEG): container finished" podID="a0cad87e-348b-45d8-a9b9-cdc83731dbd3" containerID="9d021ce8eac1ccdb63fd79a37b95b7a5c7a3840c139a374a9cf5343934bd2e86" exitCode=0 Nov 04 05:44:19.402563 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:19.402508 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr" event={"ID":"a0cad87e-348b-45d8-a9b9-cdc83731dbd3","Type":"ContainerDied","Data":"9d021ce8eac1ccdb63fd79a37b95b7a5c7a3840c139a374a9cf5343934bd2e86"} Nov 04 05:44:19.532762 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:19.532713 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8" podStartSLOduration=2.471883477 podStartE2EDuration="4.532697041s" totalImagesPullingTime="2.060813564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:44:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:44:19.530852766 +0000 UTC m=+993.410245246" watchObservedRunningTime="2026-11-04 05:44:19.532697041 +0000 UTC m=+993.412089514" Nov 04 05:44:19.588237 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:19.588211 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl" Nov 04 05:44:19.754889 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:19.754813 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f42dd6a-1809-4cb1-af96-f23fa8cb693c-kube-api-access-wwvh4\" (UniqueName: \"kubernetes.io/projected/1f42dd6a-1809-4cb1-af96-f23fa8cb693c-kube-api-access-wwvh4\") pod \"1f42dd6a-1809-4cb1-af96-f23fa8cb693c\" (UID: \"1f42dd6a-1809-4cb1-af96-f23fa8cb693c\") " Nov 04 05:44:19.754889 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:19.754877 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f42dd6a-1809-4cb1-af96-f23fa8cb693c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1f42dd6a-1809-4cb1-af96-f23fa8cb693c-maas-api-xks-ca\") pod \"1f42dd6a-1809-4cb1-af96-f23fa8cb693c\" (UID: \"1f42dd6a-1809-4cb1-af96-f23fa8cb693c\") " Nov 04 05:44:19.755087 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:19.754910 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f42dd6a-1809-4cb1-af96-f23fa8cb693c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1f42dd6a-1809-4cb1-af96-f23fa8cb693c-maas-api-openshift-ca\") pod \"1f42dd6a-1809-4cb1-af96-f23fa8cb693c\" (UID: \"1f42dd6a-1809-4cb1-af96-f23fa8cb693c\") " Nov 04 05:44:19.755283 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:19.755250 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f42dd6a-1809-4cb1-af96-f23fa8cb693c-maas-api-openshift-ca" pod "1f42dd6a-1809-4cb1-af96-f23fa8cb693c" (UID: "1f42dd6a-1809-4cb1-af96-f23fa8cb693c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:44:19.757099 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:19.757071 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f42dd6a-1809-4cb1-af96-f23fa8cb693c-kube-api-access-wwvh4" pod "1f42dd6a-1809-4cb1-af96-f23fa8cb693c" (UID: "1f42dd6a-1809-4cb1-af96-f23fa8cb693c"). InnerVolumeSpecName "kube-api-access-wwvh4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:19.757190 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:19.757100 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f42dd6a-1809-4cb1-af96-f23fa8cb693c-maas-api-xks-ca" pod "1f42dd6a-1809-4cb1-af96-f23fa8cb693c" (UID: "1f42dd6a-1809-4cb1-af96-f23fa8cb693c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:19.856656 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:19.856619 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwvh4\" (UniqueName: \"kubernetes.io/projected/1f42dd6a-1809-4cb1-af96-f23fa8cb693c-kube-api-access-wwvh4\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:19.856656 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:19.856649 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1f42dd6a-1809-4cb1-af96-f23fa8cb693c-maas-api-xks-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:19.856656 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:19.856659 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1f42dd6a-1809-4cb1-af96-f23fa8cb693c-maas-api-openshift-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:20.110471 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:20.110418 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh"] Nov 04 05:44:20.117137 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:20.117107 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8"] Nov 04 05:44:20.409799 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:20.409709 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl" Nov 04 05:44:20.409799 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:20.409721 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl" event={"ID":"1f42dd6a-1809-4cb1-af96-f23fa8cb693c","Type":"ContainerDied","Data":"1ca4712ce8788aef07a9b3ab9205d9cbfcb9096814986f1ca8c0e04a403fc4d1"} Nov 04 05:44:20.409799 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:20.409779 5258 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ca4712ce8788aef07a9b3ab9205d9cbfcb9096814986f1ca8c0e04a403fc4d1" Nov 04 05:44:20.411479 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:20.411447 5258 generic.go:356] "Generic (PLEG): container finished" podID="7ef5af96-976f-49c3-9d61-ca2cc1369bf2" containerID="5eb508184b37d1aac731b6d546fc4e622565352dd52864121d4f4a515e222614" exitCode=0 Nov 04 05:44:20.411619 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:20.411506 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8" event={"ID":"7ef5af96-976f-49c3-9d61-ca2cc1369bf2","Type":"ContainerDied","Data":"5eb508184b37d1aac731b6d546fc4e622565352dd52864121d4f4a515e222614"} Nov 04 05:44:20.571418 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:20.571392 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr" Nov 04 05:44:20.665326 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:20.665242 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0cad87e-348b-45d8-a9b9-cdc83731dbd3-kube-api-access-47zc7\" (UniqueName: \"kubernetes.io/projected/a0cad87e-348b-45d8-a9b9-cdc83731dbd3-kube-api-access-47zc7\") pod \"a0cad87e-348b-45d8-a9b9-cdc83731dbd3\" (UID: \"a0cad87e-348b-45d8-a9b9-cdc83731dbd3\") " Nov 04 05:44:20.665326 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:20.665305 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a0cad87e-348b-45d8-a9b9-cdc83731dbd3-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a0cad87e-348b-45d8-a9b9-cdc83731dbd3-maas-api-xks-ca\") pod \"a0cad87e-348b-45d8-a9b9-cdc83731dbd3\" (UID: \"a0cad87e-348b-45d8-a9b9-cdc83731dbd3\") " Nov 04 05:44:20.665531 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:20.665336 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a0cad87e-348b-45d8-a9b9-cdc83731dbd3-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a0cad87e-348b-45d8-a9b9-cdc83731dbd3-maas-api-openshift-ca\") pod \"a0cad87e-348b-45d8-a9b9-cdc83731dbd3\" (UID: \"a0cad87e-348b-45d8-a9b9-cdc83731dbd3\") " Nov 04 05:44:20.665783 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:20.665758 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0cad87e-348b-45d8-a9b9-cdc83731dbd3-maas-api-openshift-ca" pod "a0cad87e-348b-45d8-a9b9-cdc83731dbd3" (UID: "a0cad87e-348b-45d8-a9b9-cdc83731dbd3"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:44:20.665925 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:20.665910 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a0cad87e-348b-45d8-a9b9-cdc83731dbd3-maas-api-openshift-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:20.667448 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:20.667424 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0cad87e-348b-45d8-a9b9-cdc83731dbd3-maas-api-xks-ca" pod "a0cad87e-348b-45d8-a9b9-cdc83731dbd3" (UID: "a0cad87e-348b-45d8-a9b9-cdc83731dbd3"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:20.667521 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:20.667499 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0cad87e-348b-45d8-a9b9-cdc83731dbd3-kube-api-access-47zc7" pod "a0cad87e-348b-45d8-a9b9-cdc83731dbd3" (UID: "a0cad87e-348b-45d8-a9b9-cdc83731dbd3"). InnerVolumeSpecName "kube-api-access-47zc7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:20.767331 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:20.767295 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47zc7\" (UniqueName: \"kubernetes.io/projected/a0cad87e-348b-45d8-a9b9-cdc83731dbd3-kube-api-access-47zc7\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:20.767331 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:20.767326 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a0cad87e-348b-45d8-a9b9-cdc83731dbd3-maas-api-xks-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:21.419934 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:21.419903 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr" Nov 04 05:44:21.420381 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:21.419944 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr" event={"ID":"a0cad87e-348b-45d8-a9b9-cdc83731dbd3","Type":"ContainerDied","Data":"92fc51986722465bb174e9555c3b1b970b10ae5cd962b2b58341b54336ce98b0"} Nov 04 05:44:21.420381 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:21.419999 5258 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="92fc51986722465bb174e9555c3b1b970b10ae5cd962b2b58341b54336ce98b0" Nov 04 05:44:21.572958 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:21.572936 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8" Nov 04 05:44:21.678416 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:21.678327 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ef5af96-976f-49c3-9d61-ca2cc1369bf2-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ef5af96-976f-49c3-9d61-ca2cc1369bf2-maas-api-openshift-ca\") pod \"7ef5af96-976f-49c3-9d61-ca2cc1369bf2\" (UID: \"7ef5af96-976f-49c3-9d61-ca2cc1369bf2\") " Nov 04 05:44:21.678567 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:21.678454 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ef5af96-976f-49c3-9d61-ca2cc1369bf2-kube-api-access-w42cz\" (UniqueName: \"kubernetes.io/projected/7ef5af96-976f-49c3-9d61-ca2cc1369bf2-kube-api-access-w42cz\") pod \"7ef5af96-976f-49c3-9d61-ca2cc1369bf2\" (UID: \"7ef5af96-976f-49c3-9d61-ca2cc1369bf2\") " Nov 04 05:44:21.678567 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:21.678498 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ef5af96-976f-49c3-9d61-ca2cc1369bf2-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ef5af96-976f-49c3-9d61-ca2cc1369bf2-maas-api-xks-ca\") pod \"7ef5af96-976f-49c3-9d61-ca2cc1369bf2\" (UID: \"7ef5af96-976f-49c3-9d61-ca2cc1369bf2\") " Nov 04 05:44:21.678699 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:21.678676 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ef5af96-976f-49c3-9d61-ca2cc1369bf2-maas-api-openshift-ca" pod "7ef5af96-976f-49c3-9d61-ca2cc1369bf2" (UID: "7ef5af96-976f-49c3-9d61-ca2cc1369bf2"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:44:21.678873 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:21.678860 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ef5af96-976f-49c3-9d61-ca2cc1369bf2-maas-api-openshift-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:21.680587 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:21.680560 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ef5af96-976f-49c3-9d61-ca2cc1369bf2-kube-api-access-w42cz" pod "7ef5af96-976f-49c3-9d61-ca2cc1369bf2" (UID: "7ef5af96-976f-49c3-9d61-ca2cc1369bf2"). InnerVolumeSpecName "kube-api-access-w42cz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:21.680700 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:21.680595 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ef5af96-976f-49c3-9d61-ca2cc1369bf2-maas-api-xks-ca" pod "7ef5af96-976f-49c3-9d61-ca2cc1369bf2" (UID: "7ef5af96-976f-49c3-9d61-ca2cc1369bf2"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:21.780379 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:21.780342 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w42cz\" (UniqueName: \"kubernetes.io/projected/7ef5af96-976f-49c3-9d61-ca2cc1369bf2-kube-api-access-w42cz\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:21.780379 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:21.780374 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ef5af96-976f-49c3-9d61-ca2cc1369bf2-maas-api-xks-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:22.426316 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:22.426286 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8" Nov 04 05:44:22.426704 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:22.426291 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8" event={"ID":"7ef5af96-976f-49c3-9d61-ca2cc1369bf2","Type":"ContainerDied","Data":"8d0c27d9a6038188aec4663da2a8e053a22dd409b7c17b7001e383a48412da84"} Nov 04 05:44:22.426704 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:22.426408 5258 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8d0c27d9a6038188aec4663da2a8e053a22dd409b7c17b7001e383a48412da84" Nov 04 05:44:22.781818 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:44:22.781783 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:44:28.521005 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.520924 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c5ab8c96-lzr2j"] Nov 04 05:44:28.521478 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.521468 5258 cpu_manager.go:414] "containerMap: removing container" podUID="1f42dd6a-1809-4cb1-af96-f23fa8cb693c" containerName="revoke-keys" Nov 04 05:44:28.521520 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.521481 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f42dd6a-1809-4cb1-af96-f23fa8cb693c" containerName="revoke-keys" Nov 04 05:44:28.521520 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.521513 5258 cpu_manager.go:414] "containerMap: removing container" podUID="7ef5af96-976f-49c3-9d61-ca2cc1369bf2" containerName="revoke-keys" Nov 04 05:44:28.521520 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.521518 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ef5af96-976f-49c3-9d61-ca2cc1369bf2" containerName="revoke-keys" Nov 04 05:44:28.521609 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.521529 5258 cpu_manager.go:414] "containerMap: removing container" podUID="a0cad87e-348b-45d8-a9b9-cdc83731dbd3" containerName="revoke-keys" Nov 04 05:44:28.521609 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.521535 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a0cad87e-348b-45d8-a9b9-cdc83731dbd3" containerName="revoke-keys" Nov 04 05:44:28.521671 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.521626 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f42dd6a-1809-4cb1-af96-f23fa8cb693c" containerName="revoke-keys" Nov 04 05:44:28.521671 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.521638 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ef5af96-976f-49c3-9d61-ca2cc1369bf2" containerName="revoke-keys" Nov 04 05:44:28.521671 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.521645 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="a0cad87e-348b-45d8-a9b9-cdc83731dbd3" containerName="revoke-keys" Nov 04 05:44:28.525495 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.525478 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c5ab8c96-lzr2j" Nov 04 05:44:28.556227 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.556197 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6152a395-dec0-41b3-adc6-a5e7294aba66-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c5ab8c96-lzr2j\" (UID: \"6152a395-dec0-41b3-adc6-a5e7294aba66\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c5ab8c96-lzr2j" Nov 04 05:44:28.556390 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.556315 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6152a395-dec0-41b3-adc6-a5e7294aba66-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c5ab8c96-lzr2j\" (UID: \"6152a395-dec0-41b3-adc6-a5e7294aba66\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c5ab8c96-lzr2j" Nov 04 05:44:28.556390 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.556345 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85bmn\" (UniqueName: \"kubernetes.io/projected/6152a395-dec0-41b3-adc6-a5e7294aba66-kube-api-access-85bmn\") pod \"maas-api-revoke-sub-c5ab8c96-lzr2j\" (UID: \"6152a395-dec0-41b3-adc6-a5e7294aba66\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c5ab8c96-lzr2j" Nov 04 05:44:28.641362 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.641327 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Nov 04 05:44:28.641561 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.641542 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 05:44:28.641561 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.641553 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 05:44:28.641686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.641563 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dnddh\"" Nov 04 05:44:28.643873 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.643849 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c5ab8c96-lzr2j"] Nov 04 05:44:28.657010 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.656979 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6152a395-dec0-41b3-adc6-a5e7294aba66-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c5ab8c96-lzr2j\" (UID: \"6152a395-dec0-41b3-adc6-a5e7294aba66\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c5ab8c96-lzr2j" Nov 04 05:44:28.657154 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.657087 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6152a395-dec0-41b3-adc6-a5e7294aba66-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c5ab8c96-lzr2j\" (UID: \"6152a395-dec0-41b3-adc6-a5e7294aba66\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c5ab8c96-lzr2j" Nov 04 05:44:28.657154 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.657124 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85bmn\" (UniqueName: \"kubernetes.io/projected/6152a395-dec0-41b3-adc6-a5e7294aba66-kube-api-access-85bmn\") pod \"maas-api-revoke-sub-c5ab8c96-lzr2j\" (UID: \"6152a395-dec0-41b3-adc6-a5e7294aba66\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c5ab8c96-lzr2j" Nov 04 05:44:28.657778 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.657756 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6152a395-dec0-41b3-adc6-a5e7294aba66-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-c5ab8c96-lzr2j\" (UID: \"6152a395-dec0-41b3-adc6-a5e7294aba66\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c5ab8c96-lzr2j" Nov 04 05:44:28.659255 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.659237 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6152a395-dec0-41b3-adc6-a5e7294aba66-maas-api-xks-ca\") pod \"maas-api-revoke-sub-c5ab8c96-lzr2j\" (UID: \"6152a395-dec0-41b3-adc6-a5e7294aba66\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c5ab8c96-lzr2j" Nov 04 05:44:28.725601 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.725572 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85bmn\" (UniqueName: \"kubernetes.io/projected/6152a395-dec0-41b3-adc6-a5e7294aba66-kube-api-access-85bmn\") pod \"maas-api-revoke-sub-c5ab8c96-lzr2j\" (UID: \"6152a395-dec0-41b3-adc6-a5e7294aba66\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-c5ab8c96-lzr2j" Nov 04 05:44:28.838645 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:28.838558 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c5ab8c96-lzr2j" Nov 04 05:44:29.228926 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:29.228899 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c5ab8c96-lzr2j"] Nov 04 05:44:29.231456 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:44:29.231429 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6152a395_dec0_41b3_adc6_a5e7294aba66.slice/crio-66dadc2e0f3b968da028957b7f3c0a0d78d6fba736617b197d095252ed2af072 WatchSource:0}: Error finding container 66dadc2e0f3b968da028957b7f3c0a0d78d6fba736617b197d095252ed2af072: Status 404 returned error can't find the container with id 66dadc2e0f3b968da028957b7f3c0a0d78d6fba736617b197d095252ed2af072 Nov 04 05:44:29.477294 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:29.477261 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c5ab8c96-lzr2j" event={"ID":"6152a395-dec0-41b3-adc6-a5e7294aba66","Type":"ContainerStarted","Data":"66dadc2e0f3b968da028957b7f3c0a0d78d6fba736617b197d095252ed2af072"} Nov 04 05:44:30.127944 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:30.127908 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4"] Nov 04 05:44:30.131807 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:30.131789 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4" Nov 04 05:44:30.171416 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:30.171380 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e69da566-4tdk4\" (UID: \"8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4" Nov 04 05:44:30.171599 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:30.171444 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e69da566-4tdk4\" (UID: \"8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4" Nov 04 05:44:30.171678 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:30.171603 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bmg7\" (UniqueName: \"kubernetes.io/projected/8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab-kube-api-access-4bmg7\") pod \"maas-api-revoke-sub-e69da566-4tdk4\" (UID: \"8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4" Nov 04 05:44:30.221488 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:30.221456 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4"] Nov 04 05:44:30.273177 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:30.273149 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e69da566-4tdk4\" (UID: \"8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4" Nov 04 05:44:30.273330 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:30.273195 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e69da566-4tdk4\" (UID: \"8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4" Nov 04 05:44:30.273330 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:30.273248 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bmg7\" (UniqueName: \"kubernetes.io/projected/8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab-kube-api-access-4bmg7\") pod \"maas-api-revoke-sub-e69da566-4tdk4\" (UID: \"8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4" Nov 04 05:44:30.273883 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:30.273857 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e69da566-4tdk4\" (UID: \"8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4" Nov 04 05:44:30.275550 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:30.275520 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e69da566-4tdk4\" (UID: \"8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4" Nov 04 05:44:30.297424 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:30.297395 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:44:30.340764 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:30.340730 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bmg7\" (UniqueName: \"kubernetes.io/projected/8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab-kube-api-access-4bmg7\") pod \"maas-api-revoke-sub-e69da566-4tdk4\" (UID: \"8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4" Nov 04 05:44:30.484670 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:30.484580 5258 generic.go:356] "Generic (PLEG): container finished" podID="6152a395-dec0-41b3-adc6-a5e7294aba66" containerID="904938f4e3333d010cb07253e809c3eef276bf6cfc8d24fffa8ef231db4d7035" exitCode=0 Nov 04 05:44:30.484670 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:30.484634 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c5ab8c96-lzr2j" event={"ID":"6152a395-dec0-41b3-adc6-a5e7294aba66","Type":"ContainerDied","Data":"904938f4e3333d010cb07253e809c3eef276bf6cfc8d24fffa8ef231db4d7035"} Nov 04 05:44:30.502826 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:30.502794 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4" Nov 04 05:44:30.911975 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:30.911943 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4"] Nov 04 05:44:30.913086 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:44:30.913055 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d8ef11f_33d3_4fe6_b255_0f015fe2e1ab.slice/crio-a7d0aca61d8b45da172e8d48ef60442c88479c9d2b57a4d6e5e8307de03e7112 WatchSource:0}: Error finding container a7d0aca61d8b45da172e8d48ef60442c88479c9d2b57a4d6e5e8307de03e7112: Status 404 returned error can't find the container with id a7d0aca61d8b45da172e8d48ef60442c88479c9d2b57a4d6e5e8307de03e7112 Nov 04 05:44:30.951721 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:30.951689 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" podUID="07d861a6-b569-4144-83a7-6376c2ab6514" containerName="main" containerID="cri-o://f56dbc3a028421868bdf329053cd5c04d3bcd0ac0dfa3e0cd4d75f35293f43ae" gracePeriod=45 Nov 04 05:44:31.096152 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.096117 5258 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" podUID="07d861a6-b569-4144-83a7-6376c2ab6514" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": dial tcp 10.134.0.53:8000: connect: connection refused" Nov 04 05:44:31.291800 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.291768 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:44:31.499334 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.494861 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-model-cache\") pod \"07d861a6-b569-4144-83a7-6376c2ab6514\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " Nov 04 05:44:31.499334 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.494957 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/07d861a6-b569-4144-83a7-6376c2ab6514-kube-api-access-bltck\" (UniqueName: \"kubernetes.io/projected/07d861a6-b569-4144-83a7-6376c2ab6514-kube-api-access-bltck\") pod \"07d861a6-b569-4144-83a7-6376c2ab6514\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " Nov 04 05:44:31.499334 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.495007 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-home\" (UniqueName: \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-home\") pod \"07d861a6-b569-4144-83a7-6376c2ab6514\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " Nov 04 05:44:31.499334 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.495073 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-tmp-dir\") pod \"07d861a6-b569-4144-83a7-6376c2ab6514\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " Nov 04 05:44:31.499334 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.495175 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-dshm\" (UniqueName: \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-dshm\") pod \"07d861a6-b569-4144-83a7-6376c2ab6514\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " Nov 04 05:44:31.499334 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.495259 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/07d861a6-b569-4144-83a7-6376c2ab6514-tls-certs\" (UniqueName: \"kubernetes.io/secret/07d861a6-b569-4144-83a7-6376c2ab6514-tls-certs\") pod \"07d861a6-b569-4144-83a7-6376c2ab6514\" (UID: \"07d861a6-b569-4144-83a7-6376c2ab6514\") " Nov 04 05:44:31.499334 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.496645 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-home" pod "07d861a6-b569-4144-83a7-6376c2ab6514" (UID: "07d861a6-b569-4144-83a7-6376c2ab6514"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:31.499334 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.496851 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-model-cache" pod "07d861a6-b569-4144-83a7-6376c2ab6514" (UID: "07d861a6-b569-4144-83a7-6376c2ab6514"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:31.499334 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.497454 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-tmp-dir" pod "07d861a6-b569-4144-83a7-6376c2ab6514" (UID: "07d861a6-b569-4144-83a7-6376c2ab6514"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:31.500989 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.500234 5258 generic.go:356] "Generic (PLEG): container finished" podID="07d861a6-b569-4144-83a7-6376c2ab6514" containerID="f56dbc3a028421868bdf329053cd5c04d3bcd0ac0dfa3e0cd4d75f35293f43ae" exitCode=0 Nov 04 05:44:31.500989 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.500438 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" Nov 04 05:44:31.500989 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.500749 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" event={"ID":"07d861a6-b569-4144-83a7-6376c2ab6514","Type":"ContainerDied","Data":"f56dbc3a028421868bdf329053cd5c04d3bcd0ac0dfa3e0cd4d75f35293f43ae"} Nov 04 05:44:31.500989 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.500791 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9" event={"ID":"07d861a6-b569-4144-83a7-6376c2ab6514","Type":"ContainerDied","Data":"8b3dcb1f835f42d508563559885ddbfa2ef9610990db262fa9d33e0077861e45"} Nov 04 05:44:31.500989 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.500814 5258 scope.go:122] "RemoveContainer" containerID="f56dbc3a028421868bdf329053cd5c04d3bcd0ac0dfa3e0cd4d75f35293f43ae" Nov 04 05:44:31.501908 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.501872 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-dshm" pod "07d861a6-b569-4144-83a7-6376c2ab6514" (UID: "07d861a6-b569-4144-83a7-6376c2ab6514"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:31.502147 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.502123 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07d861a6-b569-4144-83a7-6376c2ab6514-kube-api-access-bltck" pod "07d861a6-b569-4144-83a7-6376c2ab6514" (UID: "07d861a6-b569-4144-83a7-6376c2ab6514"). InnerVolumeSpecName "kube-api-access-bltck". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:31.502878 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.502860 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07d861a6-b569-4144-83a7-6376c2ab6514-tls-certs" pod "07d861a6-b569-4144-83a7-6376c2ab6514" (UID: "07d861a6-b569-4144-83a7-6376c2ab6514"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:31.503740 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.503716 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4" event={"ID":"8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab","Type":"ContainerStarted","Data":"5e3ad6d274ed0ad79ed57f41642bb2c7180fd9615299f7fb91eaac51853d359d"} Nov 04 05:44:31.503833 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.503770 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4" event={"ID":"8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab","Type":"ContainerStarted","Data":"a7d0aca61d8b45da172e8d48ef60442c88479c9d2b57a4d6e5e8307de03e7112"} Nov 04 05:44:31.512364 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.512250 5258 scope.go:122] "RemoveContainer" containerID="f56dbc3a028421868bdf329053cd5c04d3bcd0ac0dfa3e0cd4d75f35293f43ae" Nov 04 05:44:31.512631 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:44:31.512607 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f56dbc3a028421868bdf329053cd5c04d3bcd0ac0dfa3e0cd4d75f35293f43ae\": container with ID starting with f56dbc3a028421868bdf329053cd5c04d3bcd0ac0dfa3e0cd4d75f35293f43ae not found: ID does not exist" containerID="f56dbc3a028421868bdf329053cd5c04d3bcd0ac0dfa3e0cd4d75f35293f43ae" Nov 04 05:44:31.512709 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.512643 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f56dbc3a028421868bdf329053cd5c04d3bcd0ac0dfa3e0cd4d75f35293f43ae"} err="failed to get container status \"f56dbc3a028421868bdf329053cd5c04d3bcd0ac0dfa3e0cd4d75f35293f43ae\": rpc error: code = NotFound desc = could not find container \"f56dbc3a028421868bdf329053cd5c04d3bcd0ac0dfa3e0cd4d75f35293f43ae\": container with ID starting with f56dbc3a028421868bdf329053cd5c04d3bcd0ac0dfa3e0cd4d75f35293f43ae not found: ID does not exist" Nov 04 05:44:31.612431 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.597573 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bltck\" (UniqueName: \"kubernetes.io/projected/07d861a6-b569-4144-83a7-6376c2ab6514-kube-api-access-bltck\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:31.612431 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.597610 5258 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-home\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:31.612431 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.597627 5258 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-tmp-dir\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:31.612431 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.597641 5258 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-dshm\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:31.612431 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.597654 5258 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07d861a6-b569-4144-83a7-6376c2ab6514-tls-certs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:31.612431 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.597668 5258 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07d861a6-b569-4144-83a7-6376c2ab6514-model-cache\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:31.626425 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.626392 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm"] Nov 04 05:44:31.694079 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.692888 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4" podStartSLOduration=1.692862506 podStartE2EDuration="1.692862506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:44:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:44:31.689945809 +0000 UTC m=+1005.569338283" watchObservedRunningTime="2026-11-04 05:44:31.692862506 +0000 UTC m=+1005.572254976" Nov 04 05:44:31.707139 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.707008 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c5ab8c96-lzr2j" Nov 04 05:44:31.759965 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.759872 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m"] Nov 04 05:44:31.799133 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.799087 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6152a395-dec0-41b3-adc6-a5e7294aba66-kube-api-access-85bmn\" (UniqueName: \"kubernetes.io/projected/6152a395-dec0-41b3-adc6-a5e7294aba66-kube-api-access-85bmn\") pod \"6152a395-dec0-41b3-adc6-a5e7294aba66\" (UID: \"6152a395-dec0-41b3-adc6-a5e7294aba66\") " Nov 04 05:44:31.799275 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.799204 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6152a395-dec0-41b3-adc6-a5e7294aba66-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6152a395-dec0-41b3-adc6-a5e7294aba66-maas-api-openshift-ca\") pod \"6152a395-dec0-41b3-adc6-a5e7294aba66\" (UID: \"6152a395-dec0-41b3-adc6-a5e7294aba66\") " Nov 04 05:44:31.799664 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.799633 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6152a395-dec0-41b3-adc6-a5e7294aba66-maas-api-openshift-ca" pod "6152a395-dec0-41b3-adc6-a5e7294aba66" (UID: "6152a395-dec0-41b3-adc6-a5e7294aba66"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:44:31.801529 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.801501 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6152a395-dec0-41b3-adc6-a5e7294aba66-kube-api-access-85bmn" pod "6152a395-dec0-41b3-adc6-a5e7294aba66" (UID: "6152a395-dec0-41b3-adc6-a5e7294aba66"). InnerVolumeSpecName "kube-api-access-85bmn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:31.900167 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.900133 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6152a395-dec0-41b3-adc6-a5e7294aba66-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6152a395-dec0-41b3-adc6-a5e7294aba66-maas-api-xks-ca\") pod \"6152a395-dec0-41b3-adc6-a5e7294aba66\" (UID: \"6152a395-dec0-41b3-adc6-a5e7294aba66\") " Nov 04 05:44:31.900515 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.900495 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6152a395-dec0-41b3-adc6-a5e7294aba66-maas-api-openshift-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:31.900515 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.900512 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85bmn\" (UniqueName: \"kubernetes.io/projected/6152a395-dec0-41b3-adc6-a5e7294aba66-kube-api-access-85bmn\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:31.902169 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:31.902138 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6152a395-dec0-41b3-adc6-a5e7294aba66-maas-api-xks-ca" pod "6152a395-dec0-41b3-adc6-a5e7294aba66" (UID: "6152a395-dec0-41b3-adc6-a5e7294aba66"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:32.001415 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:32.001382 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6152a395-dec0-41b3-adc6-a5e7294aba66-maas-api-xks-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:32.088114 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:32.079701 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9"] Nov 04 05:44:32.108069 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:32.103000 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-098178/simulator-w1-098178-kserve-78f896649-qprd9"] Nov 04 05:44:32.272556 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:32.272521 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="07d861a6-b569-4144-83a7-6376c2ab6514" path="/var/lib/kubelet/pods/07d861a6-b569-4144-83a7-6376c2ab6514/volumes" Nov 04 05:44:32.511396 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:32.511357 5258 generic.go:356] "Generic (PLEG): container finished" podID="8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab" containerID="5e3ad6d274ed0ad79ed57f41642bb2c7180fd9615299f7fb91eaac51853d359d" exitCode=0 Nov 04 05:44:32.511866 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:32.511442 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4" event={"ID":"8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab","Type":"ContainerDied","Data":"5e3ad6d274ed0ad79ed57f41642bb2c7180fd9615299f7fb91eaac51853d359d"} Nov 04 05:44:32.512793 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:32.512773 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c5ab8c96-lzr2j" Nov 04 05:44:32.512910 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:32.512814 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-c5ab8c96-lzr2j" event={"ID":"6152a395-dec0-41b3-adc6-a5e7294aba66","Type":"ContainerDied","Data":"66dadc2e0f3b968da028957b7f3c0a0d78d6fba736617b197d095252ed2af072"} Nov 04 05:44:32.512910 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:32.512856 5258 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66dadc2e0f3b968da028957b7f3c0a0d78d6fba736617b197d095252ed2af072" Nov 04 05:44:33.686741 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:33.686716 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4" Nov 04 05:44:33.717973 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:33.717944 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab-maas-api-openshift-ca\") pod \"8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab\" (UID: \"8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab\") " Nov 04 05:44:33.718146 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:33.718001 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab-maas-api-xks-ca\") pod \"8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab\" (UID: \"8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab\") " Nov 04 05:44:33.718146 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:33.718121 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab-kube-api-access-4bmg7\" (UniqueName: \"kubernetes.io/projected/8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab-kube-api-access-4bmg7\") pod \"8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab\" (UID: \"8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab\") " Nov 04 05:44:33.718391 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:33.718367 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab-maas-api-openshift-ca" pod "8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab" (UID: "8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:44:33.718581 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:33.718558 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab-maas-api-openshift-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:33.720215 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:33.720190 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab-maas-api-xks-ca" pod "8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab" (UID: "8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:33.720413 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:33.720385 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab-kube-api-access-4bmg7" pod "8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab" (UID: "8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab"). InnerVolumeSpecName "kube-api-access-4bmg7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:33.819789 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:33.819709 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab-maas-api-xks-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:33.819789 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:33.819738 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bmg7\" (UniqueName: \"kubernetes.io/projected/8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab-kube-api-access-4bmg7\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:34.528548 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:34.528409 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4" Nov 04 05:44:34.528548 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:34.528418 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4" event={"ID":"8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab","Type":"ContainerDied","Data":"a7d0aca61d8b45da172e8d48ef60442c88479c9d2b57a4d6e5e8307de03e7112"} Nov 04 05:44:34.528548 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:34.528474 5258 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a7d0aca61d8b45da172e8d48ef60442c88479c9d2b57a4d6e5e8307de03e7112" Nov 04 05:44:35.132327 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.132291 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" podUID="9e98783f-2552-48ab-b17a-8d1d0d214950" containerName="main" containerID="cri-o://bf512bc7d159d174c97eb0f7e799dec08c9f8b1e7b4d1f14832ab37abf3c0842" gracePeriod=45 Nov 04 05:44:35.136546 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.136508 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" podUID="7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3" containerName="main" containerID="cri-o://586637b8368756ae6c11bb8936aa6eab2b51e1bd5f435d2232c422c63ab234d5" gracePeriod=45 Nov 04 05:44:35.204761 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.204730 5258 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" podUID="7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": dial tcp 10.134.0.54:8000: connect: connection refused" Nov 04 05:44:35.421021 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.420995 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:44:35.424217 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.424199 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:44:35.435679 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.435661 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-model-cache\") pod \"9e98783f-2552-48ab-b17a-8d1d0d214950\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " Nov 04 05:44:35.435743 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.435702 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-dshm\" (UniqueName: \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-dshm\") pod \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " Nov 04 05:44:35.435743 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.435735 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-tmp-dir\") pod \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " Nov 04 05:44:35.435827 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.435783 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-kube-api-access-t2b7f\" (UniqueName: \"kubernetes.io/projected/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-kube-api-access-t2b7f\") pod \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " Nov 04 05:44:35.435883 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.435824 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-model-cache\") pod \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " Nov 04 05:44:35.435883 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.435856 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9e98783f-2552-48ab-b17a-8d1d0d214950-tls-certs\" (UniqueName: \"kubernetes.io/secret/9e98783f-2552-48ab-b17a-8d1d0d214950-tls-certs\") pod \"9e98783f-2552-48ab-b17a-8d1d0d214950\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " Nov 04 05:44:35.435984 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.435896 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-tls-certs\" (UniqueName: \"kubernetes.io/secret/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-tls-certs\") pod \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " Nov 04 05:44:35.435984 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.435921 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-model-cache" pod "9e98783f-2552-48ab-b17a-8d1d0d214950" (UID: "9e98783f-2552-48ab-b17a-8d1d0d214950"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:35.436108 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.436020 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-tmp-dir" pod "7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3" (UID: "7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:35.436108 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.436062 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-home\" (UniqueName: \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-home\") pod \"9e98783f-2552-48ab-b17a-8d1d0d214950\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " Nov 04 05:44:35.436212 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.436118 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-home\" (UniqueName: \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-home\") pod \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\" (UID: \"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3\") " Nov 04 05:44:35.436212 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.436143 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-model-cache" pod "7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3" (UID: "7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:35.436212 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.436195 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-dshm\") pod \"9e98783f-2552-48ab-b17a-8d1d0d214950\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " Nov 04 05:44:35.436369 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.436248 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e98783f-2552-48ab-b17a-8d1d0d214950-kube-api-access-zfs6h\" (UniqueName: \"kubernetes.io/projected/9e98783f-2552-48ab-b17a-8d1d0d214950-kube-api-access-zfs6h\") pod \"9e98783f-2552-48ab-b17a-8d1d0d214950\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " Nov 04 05:44:35.436369 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.436277 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-home" pod "9e98783f-2552-48ab-b17a-8d1d0d214950" (UID: "9e98783f-2552-48ab-b17a-8d1d0d214950"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:35.436369 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.436330 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-tmp-dir\") pod \"9e98783f-2552-48ab-b17a-8d1d0d214950\" (UID: \"9e98783f-2552-48ab-b17a-8d1d0d214950\") " Nov 04 05:44:35.436534 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.436437 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-home" pod "7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3" (UID: "7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:35.436770 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.436745 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-tmp-dir" pod "9e98783f-2552-48ab-b17a-8d1d0d214950" (UID: "9e98783f-2552-48ab-b17a-8d1d0d214950"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:35.436914 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.436877 5258 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-model-cache\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:35.436914 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.436902 5258 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-tmp-dir\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:35.437048 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.436917 5258 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-model-cache\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:35.437048 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.436931 5258 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-home\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:35.437048 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.436943 5258 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-home\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:35.437048 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.436955 5258 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-tmp-dir\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:35.438523 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.438498 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-tls-certs" pod "7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3" (UID: "7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:35.438586 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.438532 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-dshm" pod "7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3" (UID: "7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:35.438586 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.438560 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-kube-api-access-t2b7f" pod "7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3" (UID: "7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3"). InnerVolumeSpecName "kube-api-access-t2b7f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:35.438586 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.438565 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-dshm" pod "9e98783f-2552-48ab-b17a-8d1d0d214950" (UID: "9e98783f-2552-48ab-b17a-8d1d0d214950"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:35.438771 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.438754 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e98783f-2552-48ab-b17a-8d1d0d214950-tls-certs" pod "9e98783f-2552-48ab-b17a-8d1d0d214950" (UID: "9e98783f-2552-48ab-b17a-8d1d0d214950"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:35.438816 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.438788 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e98783f-2552-48ab-b17a-8d1d0d214950-kube-api-access-zfs6h" pod "9e98783f-2552-48ab-b17a-8d1d0d214950" (UID: "9e98783f-2552-48ab-b17a-8d1d0d214950"). InnerVolumeSpecName "kube-api-access-zfs6h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:35.535416 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.535379 5258 generic.go:356] "Generic (PLEG): container finished" podID="7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3" containerID="586637b8368756ae6c11bb8936aa6eab2b51e1bd5f435d2232c422c63ab234d5" exitCode=0 Nov 04 05:44:35.535598 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.535425 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" Nov 04 05:44:35.535598 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.535450 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" event={"ID":"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3","Type":"ContainerDied","Data":"586637b8368756ae6c11bb8936aa6eab2b51e1bd5f435d2232c422c63ab234d5"} Nov 04 05:44:35.535598 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.535490 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8" event={"ID":"7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3","Type":"ContainerDied","Data":"53ccb6f1726d4a43add44ab39cea7946765c635943fce44c3a051624000ebf79"} Nov 04 05:44:35.535598 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.535507 5258 scope.go:122] "RemoveContainer" containerID="586637b8368756ae6c11bb8936aa6eab2b51e1bd5f435d2232c422c63ab234d5" Nov 04 05:44:35.550071 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.540317 5258 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-dshm\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:35.550071 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.540349 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2b7f\" (UniqueName: \"kubernetes.io/projected/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-kube-api-access-t2b7f\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:35.550071 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.540367 5258 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e98783f-2552-48ab-b17a-8d1d0d214950-tls-certs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:35.550071 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.540384 5258 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3-tls-certs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:35.550071 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.540424 5258 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e98783f-2552-48ab-b17a-8d1d0d214950-dshm\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:35.550071 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.540438 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfs6h\" (UniqueName: \"kubernetes.io/projected/9e98783f-2552-48ab-b17a-8d1d0d214950-kube-api-access-zfs6h\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:35.550071 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.547297 5258 generic.go:356] "Generic (PLEG): container finished" podID="9e98783f-2552-48ab-b17a-8d1d0d214950" containerID="bf512bc7d159d174c97eb0f7e799dec08c9f8b1e7b4d1f14832ab37abf3c0842" exitCode=0 Nov 04 05:44:35.550071 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.547359 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" event={"ID":"9e98783f-2552-48ab-b17a-8d1d0d214950","Type":"ContainerDied","Data":"bf512bc7d159d174c97eb0f7e799dec08c9f8b1e7b4d1f14832ab37abf3c0842"} Nov 04 05:44:35.550071 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.547406 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" event={"ID":"9e98783f-2552-48ab-b17a-8d1d0d214950","Type":"ContainerDied","Data":"739a2f77f4a6d40e90a99145c212561d05f85a9ff7a85dbd2919a025c99bb783"} Nov 04 05:44:35.550071 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.547462 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh" Nov 04 05:44:35.554998 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.554981 5258 scope.go:122] "RemoveContainer" containerID="586637b8368756ae6c11bb8936aa6eab2b51e1bd5f435d2232c422c63ab234d5" Nov 04 05:44:35.555298 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:44:35.555279 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"586637b8368756ae6c11bb8936aa6eab2b51e1bd5f435d2232c422c63ab234d5\": container with ID starting with 586637b8368756ae6c11bb8936aa6eab2b51e1bd5f435d2232c422c63ab234d5 not found: ID does not exist" containerID="586637b8368756ae6c11bb8936aa6eab2b51e1bd5f435d2232c422c63ab234d5" Nov 04 05:44:35.555381 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.555308 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"586637b8368756ae6c11bb8936aa6eab2b51e1bd5f435d2232c422c63ab234d5"} err="failed to get container status \"586637b8368756ae6c11bb8936aa6eab2b51e1bd5f435d2232c422c63ab234d5\": rpc error: code = NotFound desc = could not find container \"586637b8368756ae6c11bb8936aa6eab2b51e1bd5f435d2232c422c63ab234d5\": container with ID starting with 586637b8368756ae6c11bb8936aa6eab2b51e1bd5f435d2232c422c63ab234d5 not found: ID does not exist" Nov 04 05:44:35.555381 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.555331 5258 scope.go:122] "RemoveContainer" containerID="bf512bc7d159d174c97eb0f7e799dec08c9f8b1e7b4d1f14832ab37abf3c0842" Nov 04 05:44:35.566152 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.566126 5258 scope.go:122] "RemoveContainer" containerID="bf512bc7d159d174c97eb0f7e799dec08c9f8b1e7b4d1f14832ab37abf3c0842" Nov 04 05:44:35.566485 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:44:35.566464 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf512bc7d159d174c97eb0f7e799dec08c9f8b1e7b4d1f14832ab37abf3c0842\": container with ID starting with bf512bc7d159d174c97eb0f7e799dec08c9f8b1e7b4d1f14832ab37abf3c0842 not found: ID does not exist" containerID="bf512bc7d159d174c97eb0f7e799dec08c9f8b1e7b4d1f14832ab37abf3c0842" Nov 04 05:44:35.566563 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.566498 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf512bc7d159d174c97eb0f7e799dec08c9f8b1e7b4d1f14832ab37abf3c0842"} err="failed to get container status \"bf512bc7d159d174c97eb0f7e799dec08c9f8b1e7b4d1f14832ab37abf3c0842\": rpc error: code = NotFound desc = could not find container \"bf512bc7d159d174c97eb0f7e799dec08c9f8b1e7b4d1f14832ab37abf3c0842\": container with ID starting with bf512bc7d159d174c97eb0f7e799dec08c9f8b1e7b4d1f14832ab37abf3c0842 not found: ID does not exist" Nov 04 05:44:35.747085 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.746996 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8"] Nov 04 05:44:35.750236 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.750211 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-90dc8d/simulator-w0-90dc8d-kserve-59896b544c-9mrw8"] Nov 04 05:44:35.931429 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.931388 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh"] Nov 04 05:44:35.937387 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:35.937358 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-90dc8d/premium-w0-90dc8d-kserve-79874fbf4b-jtgkh"] Nov 04 05:44:36.270946 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.270913 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3" path="/var/lib/kubelet/pods/7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3/volumes" Nov 04 05:44:36.271363 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.271349 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9e98783f-2552-48ab-b17a-8d1d0d214950" path="/var/lib/kubelet/pods/9e98783f-2552-48ab-b17a-8d1d0d214950/volumes" Nov 04 05:44:36.396506 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.396471 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f"] Nov 04 05:44:36.397110 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.397090 5258 cpu_manager.go:414] "containerMap: removing container" podUID="07d861a6-b569-4144-83a7-6376c2ab6514" containerName="main" Nov 04 05:44:36.397110 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.397110 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07d861a6-b569-4144-83a7-6376c2ab6514" containerName="main" Nov 04 05:44:36.397314 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.397127 5258 cpu_manager.go:414] "containerMap: removing container" podUID="9e98783f-2552-48ab-b17a-8d1d0d214950" containerName="main" Nov 04 05:44:36.397314 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.397135 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e98783f-2552-48ab-b17a-8d1d0d214950" containerName="main" Nov 04 05:44:36.397314 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.397144 5258 cpu_manager.go:414] "containerMap: removing container" podUID="8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab" containerName="revoke-keys" Nov 04 05:44:36.397314 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.397150 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab" containerName="revoke-keys" Nov 04 05:44:36.397314 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.397169 5258 cpu_manager.go:414] "containerMap: removing container" podUID="7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3" containerName="main" Nov 04 05:44:36.397314 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.397176 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3" containerName="main" Nov 04 05:44:36.397314 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.397193 5258 cpu_manager.go:414] "containerMap: removing container" podUID="6152a395-dec0-41b3-adc6-a5e7294aba66" containerName="revoke-keys" Nov 04 05:44:36.397314 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.397200 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6152a395-dec0-41b3-adc6-a5e7294aba66" containerName="revoke-keys" Nov 04 05:44:36.397674 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.397331 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="07d861a6-b569-4144-83a7-6376c2ab6514" containerName="main" Nov 04 05:44:36.397674 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.397344 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e98783f-2552-48ab-b17a-8d1d0d214950" containerName="main" Nov 04 05:44:36.397674 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.397354 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab" containerName="revoke-keys" Nov 04 05:44:36.397674 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.397365 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="7328ba4a-fae2-48c7-8d74-6dfbb0ec37e3" containerName="main" Nov 04 05:44:36.397674 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.397375 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="6152a395-dec0-41b3-adc6-a5e7294aba66" containerName="revoke-keys" Nov 04 05:44:36.402290 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.402269 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f" Nov 04 05:44:36.450674 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.450643 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwfng\" (UniqueName: \"kubernetes.io/projected/136492af-bd17-4005-96dd-4c57eebb84d9-kube-api-access-cwfng\") pod \"maas-api-revoke-sub-6e443a86-gqh8f\" (UID: \"136492af-bd17-4005-96dd-4c57eebb84d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f" Nov 04 05:44:36.450828 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.450677 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/136492af-bd17-4005-96dd-4c57eebb84d9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6e443a86-gqh8f\" (UID: \"136492af-bd17-4005-96dd-4c57eebb84d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f" Nov 04 05:44:36.450828 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.450709 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/136492af-bd17-4005-96dd-4c57eebb84d9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6e443a86-gqh8f\" (UID: \"136492af-bd17-4005-96dd-4c57eebb84d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f" Nov 04 05:44:36.464519 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.464499 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 05:44:36.464728 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.464707 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 05:44:36.464804 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.464733 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Nov 04 05:44:36.464804 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.464785 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dnddh\"" Nov 04 05:44:36.492735 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.492706 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f"] Nov 04 05:44:36.523207 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.523161 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2"] Nov 04 05:44:36.551926 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.551900 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/136492af-bd17-4005-96dd-4c57eebb84d9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6e443a86-gqh8f\" (UID: \"136492af-bd17-4005-96dd-4c57eebb84d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f" Nov 04 05:44:36.552126 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.552106 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwfng\" (UniqueName: \"kubernetes.io/projected/136492af-bd17-4005-96dd-4c57eebb84d9-kube-api-access-cwfng\") pod \"maas-api-revoke-sub-6e443a86-gqh8f\" (UID: \"136492af-bd17-4005-96dd-4c57eebb84d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f" Nov 04 05:44:36.552195 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.552146 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/136492af-bd17-4005-96dd-4c57eebb84d9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6e443a86-gqh8f\" (UID: \"136492af-bd17-4005-96dd-4c57eebb84d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f" Nov 04 05:44:36.552761 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.552743 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/136492af-bd17-4005-96dd-4c57eebb84d9-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6e443a86-gqh8f\" (UID: \"136492af-bd17-4005-96dd-4c57eebb84d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f" Nov 04 05:44:36.554405 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.554385 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/136492af-bd17-4005-96dd-4c57eebb84d9-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6e443a86-gqh8f\" (UID: \"136492af-bd17-4005-96dd-4c57eebb84d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f" Nov 04 05:44:36.620364 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.620080 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwfng\" (UniqueName: \"kubernetes.io/projected/136492af-bd17-4005-96dd-4c57eebb84d9-kube-api-access-cwfng\") pod \"maas-api-revoke-sub-6e443a86-gqh8f\" (UID: \"136492af-bd17-4005-96dd-4c57eebb84d9\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f" Nov 04 05:44:36.627555 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.627496 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm" podUID="3552c62c-8a98-4845-9817-16b284233c6f" containerName="payload-processing" containerID="cri-o://2ce5bf89b5e04041e69560a12f888374946167005c70d20158dd26c2b96d422b" gracePeriod=25 Nov 04 05:44:36.715773 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.715698 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f" Nov 04 05:44:36.761515 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:36.761465 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m" podUID="37be0c15-5de0-4b8f-aaa3-5bbe4c186f49" containerName="payload-pre-processing" containerID="cri-o://33d8d1552f01e251bdc8a1952171fec4fdabc0457f3bead08acb118ce23691cf" gracePeriod=25 Nov 04 05:44:37.008285 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:37.008208 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n"] Nov 04 05:44:37.008448 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:37.008426 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" podUID="4a15bdbb-3628-4150-92f8-8d04c5c554b4" containerName="istio-proxy" containerID="cri-o://5505c01d78631ee83602dc6d88fb84e53eca9d3e04bb2ccca1352b9e96f02c72" gracePeriod=30 Nov 04 05:44:37.010573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:37.010546 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch"] Nov 04 05:44:37.010786 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:37.010760 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" podUID="5a97e4c0-4705-445b-9fef-ef0e282cddde" containerName="istio-proxy" containerID="cri-o://14315c5cbe7073d9cfdc05599c60a018dea7802037ae3e9b31ba174b1d62d62f" gracePeriod=30 Nov 04 05:44:37.137739 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:37.137444 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f"] Nov 04 05:44:37.154574 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:44:37.154541 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod136492af_bd17_4005_96dd_4c57eebb84d9.slice/crio-e284960662a2a438afbbd431f283e1e445529cfd9273ce52800640fa99cd7e46 WatchSource:0}: Error finding container e284960662a2a438afbbd431f283e1e445529cfd9273ce52800640fa99cd7e46: Status 404 returned error can't find the container with id e284960662a2a438afbbd431f283e1e445529cfd9273ce52800640fa99cd7e46 Nov 04 05:44:37.563200 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:37.563154 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f" event={"ID":"136492af-bd17-4005-96dd-4c57eebb84d9","Type":"ContainerStarted","Data":"fa84d5381b09311450ecea4c786ae2f3e2bc1098b19b1df05dd692bc6b842612"} Nov 04 05:44:37.563591 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:37.563216 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f" event={"ID":"136492af-bd17-4005-96dd-4c57eebb84d9","Type":"ContainerStarted","Data":"e284960662a2a438afbbd431f283e1e445529cfd9273ce52800640fa99cd7e46"} Nov 04 05:44:37.692776 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:37.692722 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f" podStartSLOduration=1.692707882 podStartE2EDuration="1.692707882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:44:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:44:37.69125861 +0000 UTC m=+1011.570651082" watchObservedRunningTime="2026-11-04 05:44:37.692707882 +0000 UTC m=+1011.572100355" Nov 04 05:44:38.571192 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:38.571160 5258 generic.go:356] "Generic (PLEG): container finished" podID="136492af-bd17-4005-96dd-4c57eebb84d9" containerID="fa84d5381b09311450ecea4c786ae2f3e2bc1098b19b1df05dd692bc6b842612" exitCode=0 Nov 04 05:44:38.571561 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:38.571233 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f" event={"ID":"136492af-bd17-4005-96dd-4c57eebb84d9","Type":"ContainerDied","Data":"fa84d5381b09311450ecea4c786ae2f3e2bc1098b19b1df05dd692bc6b842612"} Nov 04 05:44:39.730321 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:39.730298 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f" Nov 04 05:44:39.791931 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:39.791901 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/136492af-bd17-4005-96dd-4c57eebb84d9-kube-api-access-cwfng\" (UniqueName: \"kubernetes.io/projected/136492af-bd17-4005-96dd-4c57eebb84d9-kube-api-access-cwfng\") pod \"136492af-bd17-4005-96dd-4c57eebb84d9\" (UID: \"136492af-bd17-4005-96dd-4c57eebb84d9\") " Nov 04 05:44:39.792100 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:39.791965 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/136492af-bd17-4005-96dd-4c57eebb84d9-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/136492af-bd17-4005-96dd-4c57eebb84d9-maas-api-openshift-ca\") pod \"136492af-bd17-4005-96dd-4c57eebb84d9\" (UID: \"136492af-bd17-4005-96dd-4c57eebb84d9\") " Nov 04 05:44:39.792146 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:39.792099 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/136492af-bd17-4005-96dd-4c57eebb84d9-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/136492af-bd17-4005-96dd-4c57eebb84d9-maas-api-xks-ca\") pod \"136492af-bd17-4005-96dd-4c57eebb84d9\" (UID: \"136492af-bd17-4005-96dd-4c57eebb84d9\") " Nov 04 05:44:39.792349 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:39.792315 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/136492af-bd17-4005-96dd-4c57eebb84d9-maas-api-openshift-ca" pod "136492af-bd17-4005-96dd-4c57eebb84d9" (UID: "136492af-bd17-4005-96dd-4c57eebb84d9"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:44:39.792468 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:39.792450 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/136492af-bd17-4005-96dd-4c57eebb84d9-maas-api-openshift-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:39.794069 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:39.794029 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/136492af-bd17-4005-96dd-4c57eebb84d9-maas-api-xks-ca" pod "136492af-bd17-4005-96dd-4c57eebb84d9" (UID: "136492af-bd17-4005-96dd-4c57eebb84d9"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:39.794156 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:39.794118 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/136492af-bd17-4005-96dd-4c57eebb84d9-kube-api-access-cwfng" pod "136492af-bd17-4005-96dd-4c57eebb84d9" (UID: "136492af-bd17-4005-96dd-4c57eebb84d9"). InnerVolumeSpecName "kube-api-access-cwfng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:39.893335 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:39.893302 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/136492af-bd17-4005-96dd-4c57eebb84d9-maas-api-xks-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:39.893335 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:39.893332 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwfng\" (UniqueName: \"kubernetes.io/projected/136492af-bd17-4005-96dd-4c57eebb84d9-kube-api-access-cwfng\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:40.408928 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:40.408892 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj"] Nov 04 05:44:40.409725 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:40.409702 5258 cpu_manager.go:414] "containerMap: removing container" podUID="136492af-bd17-4005-96dd-4c57eebb84d9" containerName="revoke-keys" Nov 04 05:44:40.409771 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:40.409731 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="136492af-bd17-4005-96dd-4c57eebb84d9" containerName="revoke-keys" Nov 04 05:44:40.409875 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:40.409863 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="136492af-bd17-4005-96dd-4c57eebb84d9" containerName="revoke-keys" Nov 04 05:44:40.423417 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:40.423386 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj" Nov 04 05:44:40.475264 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:40.475226 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj"] Nov 04 05:44:40.500196 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:40.500161 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd6p7\" (UniqueName: \"kubernetes.io/projected/6698fb4e-897c-4c3f-adc4-f2887e5ad24d-kube-api-access-pd6p7\") pod \"maas-api-revoke-sub-57776a9b-9fwsj\" (UID: \"6698fb4e-897c-4c3f-adc4-f2887e5ad24d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj" Nov 04 05:44:40.500340 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:40.500216 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6698fb4e-897c-4c3f-adc4-f2887e5ad24d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-57776a9b-9fwsj\" (UID: \"6698fb4e-897c-4c3f-adc4-f2887e5ad24d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj" Nov 04 05:44:40.500436 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:40.500408 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6698fb4e-897c-4c3f-adc4-f2887e5ad24d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-57776a9b-9fwsj\" (UID: \"6698fb4e-897c-4c3f-adc4-f2887e5ad24d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj" Nov 04 05:44:40.584907 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:40.584806 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f" event={"ID":"136492af-bd17-4005-96dd-4c57eebb84d9","Type":"ContainerDied","Data":"e284960662a2a438afbbd431f283e1e445529cfd9273ce52800640fa99cd7e46"} Nov 04 05:44:40.584907 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:40.584860 5258 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e284960662a2a438afbbd431f283e1e445529cfd9273ce52800640fa99cd7e46" Nov 04 05:44:40.584907 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:40.584817 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f" Nov 04 05:44:40.601265 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:40.601238 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6698fb4e-897c-4c3f-adc4-f2887e5ad24d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-57776a9b-9fwsj\" (UID: \"6698fb4e-897c-4c3f-adc4-f2887e5ad24d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj" Nov 04 05:44:40.601386 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:40.601329 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd6p7\" (UniqueName: \"kubernetes.io/projected/6698fb4e-897c-4c3f-adc4-f2887e5ad24d-kube-api-access-pd6p7\") pod \"maas-api-revoke-sub-57776a9b-9fwsj\" (UID: \"6698fb4e-897c-4c3f-adc4-f2887e5ad24d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj" Nov 04 05:44:40.601386 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:40.601378 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6698fb4e-897c-4c3f-adc4-f2887e5ad24d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-57776a9b-9fwsj\" (UID: \"6698fb4e-897c-4c3f-adc4-f2887e5ad24d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj" Nov 04 05:44:40.602047 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:40.602013 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6698fb4e-897c-4c3f-adc4-f2887e5ad24d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-57776a9b-9fwsj\" (UID: \"6698fb4e-897c-4c3f-adc4-f2887e5ad24d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj" Nov 04 05:44:40.603431 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:40.603400 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6698fb4e-897c-4c3f-adc4-f2887e5ad24d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-57776a9b-9fwsj\" (UID: \"6698fb4e-897c-4c3f-adc4-f2887e5ad24d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj" Nov 04 05:44:40.668687 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:40.668654 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd6p7\" (UniqueName: \"kubernetes.io/projected/6698fb4e-897c-4c3f-adc4-f2887e5ad24d-kube-api-access-pd6p7\") pod \"maas-api-revoke-sub-57776a9b-9fwsj\" (UID: \"6698fb4e-897c-4c3f-adc4-f2887e5ad24d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj" Nov 04 05:44:40.736441 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:40.736407 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj" Nov 04 05:44:41.123735 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:41.123709 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj"] Nov 04 05:44:41.126277 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:44:41.126246 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6698fb4e_897c_4c3f_adc4_f2887e5ad24d.slice/crio-098b2c71afe41c4929d0ba9ef498a72a9e9391d7c512f2341403eaf5ae25cc1c WatchSource:0}: Error finding container 098b2c71afe41c4929d0ba9ef498a72a9e9391d7c512f2341403eaf5ae25cc1c: Status 404 returned error can't find the container with id 098b2c71afe41c4929d0ba9ef498a72a9e9391d7c512f2341403eaf5ae25cc1c Nov 04 05:44:41.275883 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:41.275850 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn"] Nov 04 05:44:41.400343 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:41.400257 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt"] Nov 04 05:44:41.524006 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:41.523968 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2" podUID="63d9aa0a-c01b-46e5-b029-278cbeb206d6" containerName="payload-processing" containerID="cri-o://3b3a48ec342a44582544378c9830a0af3121aec789e6be19bbc94eaa5059f991" gracePeriod=25 Nov 04 05:44:41.592272 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:41.592236 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj" event={"ID":"6698fb4e-897c-4c3f-adc4-f2887e5ad24d","Type":"ContainerStarted","Data":"ef89ad6fff8d3aac27f372875d7afecf1d4c6c1443c9a987c458e841c0fde563"} Nov 04 05:44:41.592417 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:41.592296 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj" event={"ID":"6698fb4e-897c-4c3f-adc4-f2887e5ad24d","Type":"ContainerStarted","Data":"098b2c71afe41c4929d0ba9ef498a72a9e9391d7c512f2341403eaf5ae25cc1c"} Nov 04 05:44:42.299875 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.299847 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:44:42.303920 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.303898 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:44:42.420341 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.420311 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a15bdbb-3628-4150-92f8-8d04c5c554b4-kube-api-access-sb6kk\" (UniqueName: \"kubernetes.io/projected/4a15bdbb-3628-4150-92f8-8d04c5c554b4-kube-api-access-sb6kk\") pod \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " Nov 04 05:44:42.420341 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.420347 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a97e4c0-4705-445b-9fef-ef0e282cddde-kube-api-access-plfp4\" (UniqueName: \"kubernetes.io/projected/5a97e4c0-4705-445b-9fef-ef0e282cddde-kube-api-access-plfp4\") pod \"5a97e4c0-4705-445b-9fef-ef0e282cddde\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " Nov 04 05:44:42.420595 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.420366 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-envoy\") pod \"5a97e4c0-4705-445b-9fef-ef0e282cddde\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " Nov 04 05:44:42.420595 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.420394 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-envoy\") pod \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " Nov 04 05:44:42.420595 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.420525 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-podinfo\") pod \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " Nov 04 05:44:42.420595 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.420589 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-data\") pod \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " Nov 04 05:44:42.420796 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.420627 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-workload-certs\") pod \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " Nov 04 05:44:42.420796 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.420660 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-workload-socket\") pod \"5a97e4c0-4705-445b-9fef-ef0e282cddde\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " Nov 04 05:44:42.420796 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.420729 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istiod-ca-cert\") pod \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " Nov 04 05:44:42.420796 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.420764 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5a97e4c0-4705-445b-9fef-ef0e282cddde-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a97e4c0-4705-445b-9fef-ef0e282cddde-istiod-ca-cert\") pod \"5a97e4c0-4705-445b-9fef-ef0e282cddde\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " Nov 04 05:44:42.420985 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.420795 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-token\" (UniqueName: \"kubernetes.io/projected/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-token\") pod \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " Nov 04 05:44:42.420985 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.420927 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-workload-certs\") pod \"5a97e4c0-4705-445b-9fef-ef0e282cddde\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " Nov 04 05:44:42.421127 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.420981 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-credential-socket\") pod \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " Nov 04 05:44:42.421127 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.421009 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-workload-socket\") pod \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\" (UID: \"4a15bdbb-3628-4150-92f8-8d04c5c554b4\") " Nov 04 05:44:42.421127 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.421103 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-podinfo\") pod \"5a97e4c0-4705-445b-9fef-ef0e282cddde\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " Nov 04 05:44:42.421127 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.421099 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-workload-socket" pod "5a97e4c0-4705-445b-9fef-ef0e282cddde" (UID: "5a97e4c0-4705-445b-9fef-ef0e282cddde"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:42.421316 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.421141 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-token\" (UniqueName: \"kubernetes.io/projected/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-token\") pod \"5a97e4c0-4705-445b-9fef-ef0e282cddde\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " Nov 04 05:44:42.421316 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.421171 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-credential-socket\") pod \"5a97e4c0-4705-445b-9fef-ef0e282cddde\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " Nov 04 05:44:42.421316 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.421225 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-data\") pod \"5a97e4c0-4705-445b-9fef-ef0e282cddde\" (UID: \"5a97e4c0-4705-445b-9fef-ef0e282cddde\") " Nov 04 05:44:42.421316 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.421293 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-data" pod "4a15bdbb-3628-4150-92f8-8d04c5c554b4" (UID: "4a15bdbb-3628-4150-92f8-8d04c5c554b4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:42.421499 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.421472 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-workload-certs" pod "4a15bdbb-3628-4150-92f8-8d04c5c554b4" (UID: "4a15bdbb-3628-4150-92f8-8d04c5c554b4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:42.421711 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.421667 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-data" pod "5a97e4c0-4705-445b-9fef-ef0e282cddde" (UID: "5a97e4c0-4705-445b-9fef-ef0e282cddde"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:42.421711 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.421680 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-credential-socket" pod "4a15bdbb-3628-4150-92f8-8d04c5c554b4" (UID: "4a15bdbb-3628-4150-92f8-8d04c5c554b4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:42.422017 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.421989 5258 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-data\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:42.422017 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.422015 5258 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-data\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:42.422236 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.422048 5258 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-workload-certs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:42.422236 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.422067 5258 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-workload-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:42.422236 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.422081 5258 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-credential-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:42.422236 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.422162 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-credential-socket" pod "5a97e4c0-4705-445b-9fef-ef0e282cddde" (UID: "5a97e4c0-4705-445b-9fef-ef0e282cddde"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:42.422446 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.422250 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istiod-ca-cert" pod "4a15bdbb-3628-4150-92f8-8d04c5c554b4" (UID: "4a15bdbb-3628-4150-92f8-8d04c5c554b4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:44:42.422446 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.422308 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-workload-certs" pod "5a97e4c0-4705-445b-9fef-ef0e282cddde" (UID: "5a97e4c0-4705-445b-9fef-ef0e282cddde"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:42.422543 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.422508 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-workload-socket" pod "4a15bdbb-3628-4150-92f8-8d04c5c554b4" (UID: "4a15bdbb-3628-4150-92f8-8d04c5c554b4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:42.422613 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.422533 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a97e4c0-4705-445b-9fef-ef0e282cddde-istiod-ca-cert" pod "5a97e4c0-4705-445b-9fef-ef0e282cddde" (UID: "5a97e4c0-4705-445b-9fef-ef0e282cddde"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:44:42.423521 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.423497 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a15bdbb-3628-4150-92f8-8d04c5c554b4-kube-api-access-sb6kk" pod "4a15bdbb-3628-4150-92f8-8d04c5c554b4" (UID: "4a15bdbb-3628-4150-92f8-8d04c5c554b4"). InnerVolumeSpecName "kube-api-access-sb6kk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:42.423521 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.423513 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-envoy" pod "5a97e4c0-4705-445b-9fef-ef0e282cddde" (UID: "5a97e4c0-4705-445b-9fef-ef0e282cddde"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:42.423691 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.423512 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-envoy" pod "4a15bdbb-3628-4150-92f8-8d04c5c554b4" (UID: "4a15bdbb-3628-4150-92f8-8d04c5c554b4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:44:42.423843 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.423819 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a97e4c0-4705-445b-9fef-ef0e282cddde-kube-api-access-plfp4" pod "5a97e4c0-4705-445b-9fef-ef0e282cddde" (UID: "5a97e4c0-4705-445b-9fef-ef0e282cddde"). InnerVolumeSpecName "kube-api-access-plfp4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:42.423918 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.423895 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-podinfo" pod "4a15bdbb-3628-4150-92f8-8d04c5c554b4" (UID: "4a15bdbb-3628-4150-92f8-8d04c5c554b4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:44:42.424305 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.424287 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-podinfo" pod "5a97e4c0-4705-445b-9fef-ef0e282cddde" (UID: "5a97e4c0-4705-445b-9fef-ef0e282cddde"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:44:42.424686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.424668 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-token" pod "5a97e4c0-4705-445b-9fef-ef0e282cddde" (UID: "5a97e4c0-4705-445b-9fef-ef0e282cddde"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:42.424686 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.424678 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-token" pod "4a15bdbb-3628-4150-92f8-8d04c5c554b4" (UID: "4a15bdbb-3628-4150-92f8-8d04c5c554b4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:42.433720 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.433668 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj" podStartSLOduration=2.433652989 podStartE2EDuration="2.433652989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:44:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:44:41.738016273 +0000 UTC m=+1015.617408747" watchObservedRunningTime="2026-11-04 05:44:42.433652989 +0000 UTC m=+1016.313045457" Nov 04 05:44:42.523387 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.523352 5258 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-podinfo\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:42.523387 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.523382 5258 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-token\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:42.523387 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.523391 5258 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-credential-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:42.523609 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.523402 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sb6kk\" (UniqueName: \"kubernetes.io/projected/4a15bdbb-3628-4150-92f8-8d04c5c554b4-kube-api-access-sb6kk\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:42.523609 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.523412 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plfp4\" (UniqueName: \"kubernetes.io/projected/5a97e4c0-4705-445b-9fef-ef0e282cddde-kube-api-access-plfp4\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:42.523609 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.523421 5258 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-istio-envoy\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:42.523609 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.523430 5258 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-envoy\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:42.523609 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.523438 5258 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-podinfo\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:42.523609 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.523447 5258 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istiod-ca-cert\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:42.523609 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.523455 5258 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a97e4c0-4705-445b-9fef-ef0e282cddde-istiod-ca-cert\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:42.523609 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.523463 5258 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a15bdbb-3628-4150-92f8-8d04c5c554b4-istio-token\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:42.523609 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.523472 5258 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a97e4c0-4705-445b-9fef-ef0e282cddde-workload-certs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:42.523609 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.523479 5258 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a15bdbb-3628-4150-92f8-8d04c5c554b4-workload-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:42.600817 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.600726 5258 generic.go:356] "Generic (PLEG): container finished" podID="5a97e4c0-4705-445b-9fef-ef0e282cddde" containerID="14315c5cbe7073d9cfdc05599c60a018dea7802037ae3e9b31ba174b1d62d62f" exitCode=0 Nov 04 05:44:42.600817 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.600766 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" Nov 04 05:44:42.601042 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.600807 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" event={"ID":"5a97e4c0-4705-445b-9fef-ef0e282cddde","Type":"ContainerDied","Data":"14315c5cbe7073d9cfdc05599c60a018dea7802037ae3e9b31ba174b1d62d62f"} Nov 04 05:44:42.601042 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.600875 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch" event={"ID":"5a97e4c0-4705-445b-9fef-ef0e282cddde","Type":"ContainerDied","Data":"8c9f3985e1290fb33b6b25d3b99667b2d086604d9b9394b1ef26cb432bfbd3c5"} Nov 04 05:44:42.601042 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.600911 5258 scope.go:122] "RemoveContainer" containerID="14315c5cbe7073d9cfdc05599c60a018dea7802037ae3e9b31ba174b1d62d62f" Nov 04 05:44:42.602776 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.602752 5258 generic.go:356] "Generic (PLEG): container finished" podID="6698fb4e-897c-4c3f-adc4-f2887e5ad24d" containerID="ef89ad6fff8d3aac27f372875d7afecf1d4c6c1443c9a987c458e841c0fde563" exitCode=0 Nov 04 05:44:42.602864 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.602840 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj" event={"ID":"6698fb4e-897c-4c3f-adc4-f2887e5ad24d","Type":"ContainerDied","Data":"ef89ad6fff8d3aac27f372875d7afecf1d4c6c1443c9a987c458e841c0fde563"} Nov 04 05:44:42.604356 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.604332 5258 generic.go:356] "Generic (PLEG): container finished" podID="4a15bdbb-3628-4150-92f8-8d04c5c554b4" containerID="5505c01d78631ee83602dc6d88fb84e53eca9d3e04bb2ccca1352b9e96f02c72" exitCode=0 Nov 04 05:44:42.604455 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.604372 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" Nov 04 05:44:42.604455 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.604423 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" event={"ID":"4a15bdbb-3628-4150-92f8-8d04c5c554b4","Type":"ContainerDied","Data":"5505c01d78631ee83602dc6d88fb84e53eca9d3e04bb2ccca1352b9e96f02c72"} Nov 04 05:44:42.604526 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.604478 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n" event={"ID":"4a15bdbb-3628-4150-92f8-8d04c5c554b4","Type":"ContainerDied","Data":"e1b809a27d9f740437c7515980471b9a74367266e81021e92bc11e69a0a6fe29"} Nov 04 05:44:42.612757 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.612740 5258 scope.go:122] "RemoveContainer" containerID="14315c5cbe7073d9cfdc05599c60a018dea7802037ae3e9b31ba174b1d62d62f" Nov 04 05:44:42.613014 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:44:42.612997 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14315c5cbe7073d9cfdc05599c60a018dea7802037ae3e9b31ba174b1d62d62f\": container with ID starting with 14315c5cbe7073d9cfdc05599c60a018dea7802037ae3e9b31ba174b1d62d62f not found: ID does not exist" containerID="14315c5cbe7073d9cfdc05599c60a018dea7802037ae3e9b31ba174b1d62d62f" Nov 04 05:44:42.613097 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.613022 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14315c5cbe7073d9cfdc05599c60a018dea7802037ae3e9b31ba174b1d62d62f"} err="failed to get container status \"14315c5cbe7073d9cfdc05599c60a018dea7802037ae3e9b31ba174b1d62d62f\": rpc error: code = NotFound desc = could not find container \"14315c5cbe7073d9cfdc05599c60a018dea7802037ae3e9b31ba174b1d62d62f\": container with ID starting with 14315c5cbe7073d9cfdc05599c60a018dea7802037ae3e9b31ba174b1d62d62f not found: ID does not exist" Nov 04 05:44:42.613097 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.613057 5258 scope.go:122] "RemoveContainer" containerID="5505c01d78631ee83602dc6d88fb84e53eca9d3e04bb2ccca1352b9e96f02c72" Nov 04 05:44:42.622876 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.622857 5258 scope.go:122] "RemoveContainer" containerID="5505c01d78631ee83602dc6d88fb84e53eca9d3e04bb2ccca1352b9e96f02c72" Nov 04 05:44:42.623218 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:44:42.623195 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5505c01d78631ee83602dc6d88fb84e53eca9d3e04bb2ccca1352b9e96f02c72\": container with ID starting with 5505c01d78631ee83602dc6d88fb84e53eca9d3e04bb2ccca1352b9e96f02c72 not found: ID does not exist" containerID="5505c01d78631ee83602dc6d88fb84e53eca9d3e04bb2ccca1352b9e96f02c72" Nov 04 05:44:42.623312 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.623227 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5505c01d78631ee83602dc6d88fb84e53eca9d3e04bb2ccca1352b9e96f02c72"} err="failed to get container status \"5505c01d78631ee83602dc6d88fb84e53eca9d3e04bb2ccca1352b9e96f02c72\": rpc error: code = NotFound desc = could not find container \"5505c01d78631ee83602dc6d88fb84e53eca9d3e04bb2ccca1352b9e96f02c72\": container with ID starting with 5505c01d78631ee83602dc6d88fb84e53eca9d3e04bb2ccca1352b9e96f02c72 not found: ID does not exist" Nov 04 05:44:42.785083 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.785048 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n"] Nov 04 05:44:42.792127 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.792096 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-vmk4n"] Nov 04 05:44:42.917471 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.917440 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch"] Nov 04 05:44:42.923899 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:42.923873 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-778b7b5c47-xbdch"] Nov 04 05:44:43.515135 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:43.515100 5258 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2" podUID="63d9aa0a-c01b-46e5-b029-278cbeb206d6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 05:44:43.759298 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:43.759270 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj" Nov 04 05:44:43.837566 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:43.837478 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6698fb4e-897c-4c3f-adc4-f2887e5ad24d-kube-api-access-pd6p7\" (UniqueName: \"kubernetes.io/projected/6698fb4e-897c-4c3f-adc4-f2887e5ad24d-kube-api-access-pd6p7\") pod \"6698fb4e-897c-4c3f-adc4-f2887e5ad24d\" (UID: \"6698fb4e-897c-4c3f-adc4-f2887e5ad24d\") " Nov 04 05:44:43.837731 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:43.837597 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6698fb4e-897c-4c3f-adc4-f2887e5ad24d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6698fb4e-897c-4c3f-adc4-f2887e5ad24d-maas-api-openshift-ca\") pod \"6698fb4e-897c-4c3f-adc4-f2887e5ad24d\" (UID: \"6698fb4e-897c-4c3f-adc4-f2887e5ad24d\") " Nov 04 05:44:43.837731 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:43.837673 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6698fb4e-897c-4c3f-adc4-f2887e5ad24d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6698fb4e-897c-4c3f-adc4-f2887e5ad24d-maas-api-xks-ca\") pod \"6698fb4e-897c-4c3f-adc4-f2887e5ad24d\" (UID: \"6698fb4e-897c-4c3f-adc4-f2887e5ad24d\") " Nov 04 05:44:43.837989 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:43.837965 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6698fb4e-897c-4c3f-adc4-f2887e5ad24d-maas-api-openshift-ca" pod "6698fb4e-897c-4c3f-adc4-f2887e5ad24d" (UID: "6698fb4e-897c-4c3f-adc4-f2887e5ad24d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:44:43.839473 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:43.839451 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6698fb4e-897c-4c3f-adc4-f2887e5ad24d-kube-api-access-pd6p7" pod "6698fb4e-897c-4c3f-adc4-f2887e5ad24d" (UID: "6698fb4e-897c-4c3f-adc4-f2887e5ad24d"). InnerVolumeSpecName "kube-api-access-pd6p7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:43.839651 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:43.839630 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6698fb4e-897c-4c3f-adc4-f2887e5ad24d-maas-api-xks-ca" pod "6698fb4e-897c-4c3f-adc4-f2887e5ad24d" (UID: "6698fb4e-897c-4c3f-adc4-f2887e5ad24d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:44:43.941314 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:43.938861 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx"] Nov 04 05:44:43.941314 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:43.940217 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6698fb4e-897c-4c3f-adc4-f2887e5ad24d-maas-api-xks-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:43.941314 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:43.940244 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pd6p7\" (UniqueName: \"kubernetes.io/projected/6698fb4e-897c-4c3f-adc4-f2887e5ad24d-kube-api-access-pd6p7\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:43.941314 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:43.940271 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6698fb4e-897c-4c3f-adc4-f2887e5ad24d-maas-api-openshift-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:44.271956 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:44.271921 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4a15bdbb-3628-4150-92f8-8d04c5c554b4" path="/var/lib/kubelet/pods/4a15bdbb-3628-4150-92f8-8d04c5c554b4/volumes" Nov 04 05:44:44.272613 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:44.272583 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5a97e4c0-4705-445b-9fef-ef0e282cddde" path="/var/lib/kubelet/pods/5a97e4c0-4705-445b-9fef-ef0e282cddde/volumes" Nov 04 05:44:44.618986 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:44.618959 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj" Nov 04 05:44:44.619385 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:44.618993 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj" event={"ID":"6698fb4e-897c-4c3f-adc4-f2887e5ad24d","Type":"ContainerDied","Data":"098b2c71afe41c4929d0ba9ef498a72a9e9391d7c512f2341403eaf5ae25cc1c"} Nov 04 05:44:44.619385 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:44.619064 5258 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="098b2c71afe41c4929d0ba9ef498a72a9e9391d7c512f2341403eaf5ae25cc1c" Nov 04 05:44:46.276824 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:46.276787 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn" podUID="a6634277-09ef-4074-9aab-36e8e10155d5" containerName="payload-processing" containerID="cri-o://a3d905b2155cb14412a920bfa51a8a97b78827e114f0799aa3cf1d71af491f9e" gracePeriod=25 Nov 04 05:44:46.341144 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:46.341109 5258 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm" podUID="3552c62c-8a98-4845-9817-16b284233c6f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 05:44:46.341144 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:46.341132 5258 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m" podUID="37be0c15-5de0-4b8f-aaa3-5bbe4c186f49" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 05:44:46.401445 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:46.401407 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt" podUID="775cd728-84a4-43a3-b3a1-2fdcbb19203e" containerName="payload-pre-processing" containerID="cri-o://1f86f6ed357fdfcdfc354387501d04f8982eef96b928ba591fc7234b01f42d0b" gracePeriod=25 Nov 04 05:44:46.533630 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:46.533574 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn" Nov 04 05:44:46.635100 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:46.635066 5258 generic.go:356] "Generic (PLEG): container finished" podID="a6634277-09ef-4074-9aab-36e8e10155d5" containerID="a3d905b2155cb14412a920bfa51a8a97b78827e114f0799aa3cf1d71af491f9e" exitCode=0 Nov 04 05:44:46.635276 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:46.635109 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn" Nov 04 05:44:46.635276 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:46.635150 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn" event={"ID":"a6634277-09ef-4074-9aab-36e8e10155d5","Type":"ContainerDied","Data":"a3d905b2155cb14412a920bfa51a8a97b78827e114f0799aa3cf1d71af491f9e"} Nov 04 05:44:46.635276 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:46.635210 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn" event={"ID":"a6634277-09ef-4074-9aab-36e8e10155d5","Type":"ContainerDied","Data":"472d2ca920781808f6d4fc9db5f6410bfb0de70c9fe04e40080a5255da81276e"} Nov 04 05:44:46.635276 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:46.635245 5258 scope.go:122] "RemoveContainer" containerID="a3d905b2155cb14412a920bfa51a8a97b78827e114f0799aa3cf1d71af491f9e" Nov 04 05:44:46.643899 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:46.643884 5258 scope.go:122] "RemoveContainer" containerID="a3d905b2155cb14412a920bfa51a8a97b78827e114f0799aa3cf1d71af491f9e" Nov 04 05:44:46.644166 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:44:46.644145 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3d905b2155cb14412a920bfa51a8a97b78827e114f0799aa3cf1d71af491f9e\": container with ID starting with a3d905b2155cb14412a920bfa51a8a97b78827e114f0799aa3cf1d71af491f9e not found: ID does not exist" containerID="a3d905b2155cb14412a920bfa51a8a97b78827e114f0799aa3cf1d71af491f9e" Nov 04 05:44:46.644226 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:46.644173 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3d905b2155cb14412a920bfa51a8a97b78827e114f0799aa3cf1d71af491f9e"} err="failed to get container status \"a3d905b2155cb14412a920bfa51a8a97b78827e114f0799aa3cf1d71af491f9e\": rpc error: code = NotFound desc = could not find container \"a3d905b2155cb14412a920bfa51a8a97b78827e114f0799aa3cf1d71af491f9e\": container with ID starting with a3d905b2155cb14412a920bfa51a8a97b78827e114f0799aa3cf1d71af491f9e not found: ID does not exist" Nov 04 05:44:46.666932 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:46.666900 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a6634277-09ef-4074-9aab-36e8e10155d5-kube-api-access-zvgbh\" (UniqueName: \"kubernetes.io/projected/a6634277-09ef-4074-9aab-36e8e10155d5-kube-api-access-zvgbh\") pod \"a6634277-09ef-4074-9aab-36e8e10155d5\" (UID: \"a6634277-09ef-4074-9aab-36e8e10155d5\") " Nov 04 05:44:46.667131 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:46.666970 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a6634277-09ef-4074-9aab-36e8e10155d5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6634277-09ef-4074-9aab-36e8e10155d5-plugins-config-volume\") pod \"a6634277-09ef-4074-9aab-36e8e10155d5\" (UID: \"a6634277-09ef-4074-9aab-36e8e10155d5\") " Nov 04 05:44:46.667414 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:46.667388 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6634277-09ef-4074-9aab-36e8e10155d5-plugins-config-volume" pod "a6634277-09ef-4074-9aab-36e8e10155d5" (UID: "a6634277-09ef-4074-9aab-36e8e10155d5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:44:46.669153 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:46.669125 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6634277-09ef-4074-9aab-36e8e10155d5-kube-api-access-zvgbh" pod "a6634277-09ef-4074-9aab-36e8e10155d5" (UID: "a6634277-09ef-4074-9aab-36e8e10155d5"). InnerVolumeSpecName "kube-api-access-zvgbh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:46.768668 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:46.768637 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvgbh\" (UniqueName: \"kubernetes.io/projected/a6634277-09ef-4074-9aab-36e8e10155d5-kube-api-access-zvgbh\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:46.768668 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:46.768666 5258 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a6634277-09ef-4074-9aab-36e8e10155d5-plugins-config-volume\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:47.069204 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:47.069162 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn"] Nov 04 05:44:47.073273 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:47.073243 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-90dc8d-665d558cfd-gvxfn"] Nov 04 05:44:48.057683 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.057653 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242"] Nov 04 05:44:48.058145 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.058123 5258 cpu_manager.go:414] "containerMap: removing container" podUID="a6634277-09ef-4074-9aab-36e8e10155d5" containerName="payload-processing" Nov 04 05:44:48.058145 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.058139 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a6634277-09ef-4074-9aab-36e8e10155d5" containerName="payload-processing" Nov 04 05:44:48.058145 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.058152 5258 cpu_manager.go:414] "containerMap: removing container" podUID="6698fb4e-897c-4c3f-adc4-f2887e5ad24d" containerName="revoke-keys" Nov 04 05:44:48.058324 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.058158 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6698fb4e-897c-4c3f-adc4-f2887e5ad24d" containerName="revoke-keys" Nov 04 05:44:48.058324 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.058170 5258 cpu_manager.go:414] "containerMap: removing container" podUID="5a97e4c0-4705-445b-9fef-ef0e282cddde" containerName="istio-proxy" Nov 04 05:44:48.058324 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.058175 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a97e4c0-4705-445b-9fef-ef0e282cddde" containerName="istio-proxy" Nov 04 05:44:48.058324 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.058196 5258 cpu_manager.go:414] "containerMap: removing container" podUID="4a15bdbb-3628-4150-92f8-8d04c5c554b4" containerName="istio-proxy" Nov 04 05:44:48.058324 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.058201 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a15bdbb-3628-4150-92f8-8d04c5c554b4" containerName="istio-proxy" Nov 04 05:44:48.058324 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.058293 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a15bdbb-3628-4150-92f8-8d04c5c554b4" containerName="istio-proxy" Nov 04 05:44:48.058324 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.058305 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="a6634277-09ef-4074-9aab-36e8e10155d5" containerName="payload-processing" Nov 04 05:44:48.058324 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.058312 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="6698fb4e-897c-4c3f-adc4-f2887e5ad24d" containerName="revoke-keys" Nov 04 05:44:48.058324 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.058318 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a97e4c0-4705-445b-9fef-ef0e282cddde" containerName="istio-proxy" Nov 04 05:44:48.062904 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.062888 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:48.146515 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.146485 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242"] Nov 04 05:44:48.179452 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.179418 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-a84d48\"/\"distinct-2-w3-a84d48-kserve-self-signed-certs\"" Nov 04 05:44:48.182933 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.182908 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-dshm\") pod \"distinct-2-w3-a84d48-kserve-667694b496-lw242\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:48.183082 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.182948 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-home\") pod \"distinct-2-w3-a84d48-kserve-667694b496-lw242\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:48.183154 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.183086 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-tmp-dir\") pod \"distinct-2-w3-a84d48-kserve-667694b496-lw242\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:48.183206 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.183185 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-model-cache\") pod \"distinct-2-w3-a84d48-kserve-667694b496-lw242\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:48.183256 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.183215 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9f3696d-d689-431f-8da6-a0e2ce5039a1-tls-certs\") pod \"distinct-2-w3-a84d48-kserve-667694b496-lw242\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:48.183256 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.183233 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r26dv\" (UniqueName: \"kubernetes.io/projected/f9f3696d-d689-431f-8da6-a0e2ce5039a1-kube-api-access-r26dv\") pod \"distinct-2-w3-a84d48-kserve-667694b496-lw242\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:48.270405 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.270367 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a6634277-09ef-4074-9aab-36e8e10155d5" path="/var/lib/kubelet/pods/a6634277-09ef-4074-9aab-36e8e10155d5/volumes" Nov 04 05:44:48.284720 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.284689 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9f3696d-d689-431f-8da6-a0e2ce5039a1-tls-certs\") pod \"distinct-2-w3-a84d48-kserve-667694b496-lw242\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:48.284720 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.284724 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r26dv\" (UniqueName: \"kubernetes.io/projected/f9f3696d-d689-431f-8da6-a0e2ce5039a1-kube-api-access-r26dv\") pod \"distinct-2-w3-a84d48-kserve-667694b496-lw242\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:48.284927 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.284804 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-dshm\") pod \"distinct-2-w3-a84d48-kserve-667694b496-lw242\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:48.284927 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.284834 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-home\") pod \"distinct-2-w3-a84d48-kserve-667694b496-lw242\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:48.284927 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.284896 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-tmp-dir\") pod \"distinct-2-w3-a84d48-kserve-667694b496-lw242\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:48.285130 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.284966 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-model-cache\") pod \"distinct-2-w3-a84d48-kserve-667694b496-lw242\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:48.285366 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.285332 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-home\") pod \"distinct-2-w3-a84d48-kserve-667694b496-lw242\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:48.285483 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.285340 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-model-cache\") pod \"distinct-2-w3-a84d48-kserve-667694b496-lw242\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:48.285483 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.285397 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-tmp-dir\") pod \"distinct-2-w3-a84d48-kserve-667694b496-lw242\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:48.287085 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.287063 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-dshm\") pod \"distinct-2-w3-a84d48-kserve-667694b496-lw242\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:48.287461 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.287443 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9f3696d-d689-431f-8da6-a0e2ce5039a1-tls-certs\") pod \"distinct-2-w3-a84d48-kserve-667694b496-lw242\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:48.352861 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.352788 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r26dv\" (UniqueName: \"kubernetes.io/projected/f9f3696d-d689-431f-8da6-a0e2ce5039a1-kube-api-access-r26dv\") pod \"distinct-2-w3-a84d48-kserve-667694b496-lw242\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:48.374654 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.374621 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:48.768821 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.768673 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242"] Nov 04 05:44:48.771691 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:44:48.771660 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9f3696d_d689_431f_8da6_a0e2ce5039a1.slice/crio-9eb9318d8e61ae1d7a676d91e960a34880349a309aee3225fc4a39dd0dea9106 WatchSource:0}: Error finding container 9eb9318d8e61ae1d7a676d91e960a34880349a309aee3225fc4a39dd0dea9106: Status 404 returned error can't find the container with id 9eb9318d8e61ae1d7a676d91e960a34880349a309aee3225fc4a39dd0dea9106 Nov 04 05:44:48.939476 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:48.939435 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx" podUID="c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1" containerName="payload-processing" containerID="cri-o://6462374c1c0fa5d9b51e03551d12db632adad4541d30cb5e064c34dae6804cf0" gracePeriod=25 Nov 04 05:44:49.181454 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:49.181429 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx" Nov 04 05:44:49.292788 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:49.292750 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1-plugins-config-volume\") pod \"c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1\" (UID: \"c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1\") " Nov 04 05:44:49.292955 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:49.292834 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1-kube-api-access-psxfq\" (UniqueName: \"kubernetes.io/projected/c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1-kube-api-access-psxfq\") pod \"c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1\" (UID: \"c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1\") " Nov 04 05:44:49.293262 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:49.293236 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1-plugins-config-volume" pod "c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1" (UID: "c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:44:49.294897 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:49.294878 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1-kube-api-access-psxfq" pod "c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1" (UID: "c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1"). InnerVolumeSpecName "kube-api-access-psxfq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:49.394598 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:49.394564 5258 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1-plugins-config-volume\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:49.394598 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:49.394593 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psxfq\" (UniqueName: \"kubernetes.io/projected/c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1-kube-api-access-psxfq\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:49.656361 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:49.656320 5258 generic.go:356] "Generic (PLEG): container finished" podID="c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1" containerID="6462374c1c0fa5d9b51e03551d12db632adad4541d30cb5e064c34dae6804cf0" exitCode=0 Nov 04 05:44:49.656698 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:49.656409 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx" Nov 04 05:44:49.656698 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:49.656404 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx" event={"ID":"c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1","Type":"ContainerDied","Data":"6462374c1c0fa5d9b51e03551d12db632adad4541d30cb5e064c34dae6804cf0"} Nov 04 05:44:49.656861 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:49.656842 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx" event={"ID":"c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1","Type":"ContainerDied","Data":"451a67058dd80233b3f7d8ba1812bf08a2cf09c3cbdd6542e788e940e1ecb365"} Nov 04 05:44:49.656966 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:49.656951 5258 scope.go:122] "RemoveContainer" containerID="6462374c1c0fa5d9b51e03551d12db632adad4541d30cb5e064c34dae6804cf0" Nov 04 05:44:49.658355 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:49.658328 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" event={"ID":"f9f3696d-d689-431f-8da6-a0e2ce5039a1","Type":"ContainerStarted","Data":"1b7d4b53437b658700dcaaae12eefc1d5bde06517abce54aef1df50b1b4d6788"} Nov 04 05:44:49.658433 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:49.658363 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" event={"ID":"f9f3696d-d689-431f-8da6-a0e2ce5039a1","Type":"ContainerStarted","Data":"9eb9318d8e61ae1d7a676d91e960a34880349a309aee3225fc4a39dd0dea9106"} Nov 04 05:44:49.667101 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:49.667081 5258 scope.go:122] "RemoveContainer" containerID="6462374c1c0fa5d9b51e03551d12db632adad4541d30cb5e064c34dae6804cf0" Nov 04 05:44:49.667350 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:44:49.667330 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6462374c1c0fa5d9b51e03551d12db632adad4541d30cb5e064c34dae6804cf0\": container with ID starting with 6462374c1c0fa5d9b51e03551d12db632adad4541d30cb5e064c34dae6804cf0 not found: ID does not exist" containerID="6462374c1c0fa5d9b51e03551d12db632adad4541d30cb5e064c34dae6804cf0" Nov 04 05:44:49.667414 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:49.667356 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6462374c1c0fa5d9b51e03551d12db632adad4541d30cb5e064c34dae6804cf0"} err="failed to get container status \"6462374c1c0fa5d9b51e03551d12db632adad4541d30cb5e064c34dae6804cf0\": rpc error: code = NotFound desc = could not find container \"6462374c1c0fa5d9b51e03551d12db632adad4541d30cb5e064c34dae6804cf0\": container with ID starting with 6462374c1c0fa5d9b51e03551d12db632adad4541d30cb5e064c34dae6804cf0 not found: ID does not exist" Nov 04 05:44:49.843149 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:49.843114 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx"] Nov 04 05:44:49.846292 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:49.846266 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w6-fbbad9-d6f88b5fd-wsdqx"] Nov 04 05:44:49.972265 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:49.972164 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" podStartSLOduration=1.972149248 podStartE2EDuration="1.972149248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:44:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:44:49.971467199 +0000 UTC m=+1023.850859671" watchObservedRunningTime="2026-11-04 05:44:49.972149248 +0000 UTC m=+1023.851541760" Nov 04 05:44:50.270923 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:50.270845 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1" path="/var/lib/kubelet/pods/c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1/volumes" Nov 04 05:44:50.374884 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:50.374854 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:52.394396 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:52.394359 5258 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt" podUID="775cd728-84a4-43a3-b3a1-2fdcbb19203e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 05:44:52.795875 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:44:52.795781 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:44:53.515185 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:53.515144 5258 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2" podUID="63d9aa0a-c01b-46e5-b029-278cbeb206d6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 05:44:56.341075 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:56.341021 5258 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm" podUID="3552c62c-8a98-4845-9817-16b284233c6f" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 05:44:56.341471 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:56.341016 5258 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m" podUID="37be0c15-5de0-4b8f-aaa3-5bbe4c186f49" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 05:44:56.890167 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:56.890113 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm" Nov 04 05:44:56.977135 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:56.976934 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3552c62c-8a98-4845-9817-16b284233c6f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3552c62c-8a98-4845-9817-16b284233c6f-plugins-config-volume\") pod \"3552c62c-8a98-4845-9817-16b284233c6f\" (UID: \"3552c62c-8a98-4845-9817-16b284233c6f\") " Nov 04 05:44:56.977294 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:56.977156 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3552c62c-8a98-4845-9817-16b284233c6f-kube-api-access-vpqf6\" (UniqueName: \"kubernetes.io/projected/3552c62c-8a98-4845-9817-16b284233c6f-kube-api-access-vpqf6\") pod \"3552c62c-8a98-4845-9817-16b284233c6f\" (UID: \"3552c62c-8a98-4845-9817-16b284233c6f\") " Nov 04 05:44:56.977369 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:56.977290 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3552c62c-8a98-4845-9817-16b284233c6f-plugins-config-volume" pod "3552c62c-8a98-4845-9817-16b284233c6f" (UID: "3552c62c-8a98-4845-9817-16b284233c6f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:44:56.977603 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:56.977582 5258 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3552c62c-8a98-4845-9817-16b284233c6f-plugins-config-volume\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:56.980411 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:56.980385 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3552c62c-8a98-4845-9817-16b284233c6f-kube-api-access-vpqf6" pod "3552c62c-8a98-4845-9817-16b284233c6f" (UID: "3552c62c-8a98-4845-9817-16b284233c6f"). InnerVolumeSpecName "kube-api-access-vpqf6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:57.006569 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.006543 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m" Nov 04 05:44:57.078685 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.078650 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37be0c15-5de0-4b8f-aaa3-5bbe4c186f49-kube-api-access-ndc4m\" (UniqueName: \"kubernetes.io/projected/37be0c15-5de0-4b8f-aaa3-5bbe4c186f49-kube-api-access-ndc4m\") pod \"37be0c15-5de0-4b8f-aaa3-5bbe4c186f49\" (UID: \"37be0c15-5de0-4b8f-aaa3-5bbe4c186f49\") " Nov 04 05:44:57.078856 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.078695 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/37be0c15-5de0-4b8f-aaa3-5bbe4c186f49-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37be0c15-5de0-4b8f-aaa3-5bbe4c186f49-plugins-config-volume\") pod \"37be0c15-5de0-4b8f-aaa3-5bbe4c186f49\" (UID: \"37be0c15-5de0-4b8f-aaa3-5bbe4c186f49\") " Nov 04 05:44:57.079089 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.079069 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpqf6\" (UniqueName: \"kubernetes.io/projected/3552c62c-8a98-4845-9817-16b284233c6f-kube-api-access-vpqf6\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:57.079172 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.079155 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37be0c15-5de0-4b8f-aaa3-5bbe4c186f49-plugins-config-volume" pod "37be0c15-5de0-4b8f-aaa3-5bbe4c186f49" (UID: "37be0c15-5de0-4b8f-aaa3-5bbe4c186f49"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:44:57.080801 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.080779 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37be0c15-5de0-4b8f-aaa3-5bbe4c186f49-kube-api-access-ndc4m" pod "37be0c15-5de0-4b8f-aaa3-5bbe4c186f49" (UID: "37be0c15-5de0-4b8f-aaa3-5bbe4c186f49"). InnerVolumeSpecName "kube-api-access-ndc4m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:44:57.180022 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.179931 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndc4m\" (UniqueName: \"kubernetes.io/projected/37be0c15-5de0-4b8f-aaa3-5bbe4c186f49-kube-api-access-ndc4m\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:57.180022 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.179962 5258 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37be0c15-5de0-4b8f-aaa3-5bbe4c186f49-plugins-config-volume\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:44:57.710697 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.710664 5258 generic.go:356] "Generic (PLEG): container finished" podID="37be0c15-5de0-4b8f-aaa3-5bbe4c186f49" containerID="33d8d1552f01e251bdc8a1952171fec4fdabc0457f3bead08acb118ce23691cf" exitCode=0 Nov 04 05:44:57.711140 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.710725 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m" Nov 04 05:44:57.711140 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.710714 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m" event={"ID":"37be0c15-5de0-4b8f-aaa3-5bbe4c186f49","Type":"ContainerDied","Data":"33d8d1552f01e251bdc8a1952171fec4fdabc0457f3bead08acb118ce23691cf"} Nov 04 05:44:57.711140 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.710848 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m" event={"ID":"37be0c15-5de0-4b8f-aaa3-5bbe4c186f49","Type":"ContainerDied","Data":"c613ea9f5abb6895ae8003f7c0046d67dac5cf684b3d9ec896dc2cccfe66ab05"} Nov 04 05:44:57.711140 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.710880 5258 scope.go:122] "RemoveContainer" containerID="33d8d1552f01e251bdc8a1952171fec4fdabc0457f3bead08acb118ce23691cf" Nov 04 05:44:57.712349 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.712324 5258 generic.go:356] "Generic (PLEG): container finished" podID="3552c62c-8a98-4845-9817-16b284233c6f" containerID="2ce5bf89b5e04041e69560a12f888374946167005c70d20158dd26c2b96d422b" exitCode=0 Nov 04 05:44:57.712451 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.712373 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm" Nov 04 05:44:57.712451 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.712403 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm" event={"ID":"3552c62c-8a98-4845-9817-16b284233c6f","Type":"ContainerDied","Data":"2ce5bf89b5e04041e69560a12f888374946167005c70d20158dd26c2b96d422b"} Nov 04 05:44:57.712560 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.712457 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm" event={"ID":"3552c62c-8a98-4845-9817-16b284233c6f","Type":"ContainerDied","Data":"77078d38b1fe9daea57779971d37dd95a6ea3a70bcffde6558fa4d18e3addd28"} Nov 04 05:44:57.721621 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.721602 5258 scope.go:122] "RemoveContainer" containerID="33d8d1552f01e251bdc8a1952171fec4fdabc0457f3bead08acb118ce23691cf" Nov 04 05:44:57.721927 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:44:57.721903 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33d8d1552f01e251bdc8a1952171fec4fdabc0457f3bead08acb118ce23691cf\": container with ID starting with 33d8d1552f01e251bdc8a1952171fec4fdabc0457f3bead08acb118ce23691cf not found: ID does not exist" containerID="33d8d1552f01e251bdc8a1952171fec4fdabc0457f3bead08acb118ce23691cf" Nov 04 05:44:57.722069 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.721931 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33d8d1552f01e251bdc8a1952171fec4fdabc0457f3bead08acb118ce23691cf"} err="failed to get container status \"33d8d1552f01e251bdc8a1952171fec4fdabc0457f3bead08acb118ce23691cf\": rpc error: code = NotFound desc = could not find container \"33d8d1552f01e251bdc8a1952171fec4fdabc0457f3bead08acb118ce23691cf\": container with ID starting with 33d8d1552f01e251bdc8a1952171fec4fdabc0457f3bead08acb118ce23691cf not found: ID does not exist" Nov 04 05:44:57.722069 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.721948 5258 scope.go:122] "RemoveContainer" containerID="2ce5bf89b5e04041e69560a12f888374946167005c70d20158dd26c2b96d422b" Nov 04 05:44:57.730976 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.730956 5258 scope.go:122] "RemoveContainer" containerID="2ce5bf89b5e04041e69560a12f888374946167005c70d20158dd26c2b96d422b" Nov 04 05:44:57.731301 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:44:57.731281 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ce5bf89b5e04041e69560a12f888374946167005c70d20158dd26c2b96d422b\": container with ID starting with 2ce5bf89b5e04041e69560a12f888374946167005c70d20158dd26c2b96d422b not found: ID does not exist" containerID="2ce5bf89b5e04041e69560a12f888374946167005c70d20158dd26c2b96d422b" Nov 04 05:44:57.731362 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.731314 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ce5bf89b5e04041e69560a12f888374946167005c70d20158dd26c2b96d422b"} err="failed to get container status \"2ce5bf89b5e04041e69560a12f888374946167005c70d20158dd26c2b96d422b\": rpc error: code = NotFound desc = could not find container \"2ce5bf89b5e04041e69560a12f888374946167005c70d20158dd26c2b96d422b\": container with ID starting with 2ce5bf89b5e04041e69560a12f888374946167005c70d20158dd26c2b96d422b not found: ID does not exist" Nov 04 05:44:57.894279 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.894248 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm"] Nov 04 05:44:57.900910 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:57.900832 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-098178-5bb799977d-r46pm"] Nov 04 05:44:58.025560 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:58.025524 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m"] Nov 04 05:44:58.029384 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:58.029355 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-098178-6b4f56d54c-nk29m"] Nov 04 05:44:58.270292 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:58.270203 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3552c62c-8a98-4845-9817-16b284233c6f" path="/var/lib/kubelet/pods/3552c62c-8a98-4845-9817-16b284233c6f/volumes" Nov 04 05:44:58.270692 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:58.270674 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="37be0c15-5de0-4b8f-aaa3-5bbe4c186f49" path="/var/lib/kubelet/pods/37be0c15-5de0-4b8f-aaa3-5bbe4c186f49/volumes" Nov 04 05:44:58.374912 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:58.374874 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:58.388791 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:58.388760 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:44:58.731615 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:44:58.731578 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:45:00.179278 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.179244 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4"] Nov 04 05:45:00.179883 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.179719 5258 cpu_manager.go:414] "containerMap: removing container" podUID="37be0c15-5de0-4b8f-aaa3-5bbe4c186f49" containerName="payload-pre-processing" Nov 04 05:45:00.179883 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.179732 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="37be0c15-5de0-4b8f-aaa3-5bbe4c186f49" containerName="payload-pre-processing" Nov 04 05:45:00.179883 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.179745 5258 cpu_manager.go:414] "containerMap: removing container" podUID="3552c62c-8a98-4845-9817-16b284233c6f" containerName="payload-processing" Nov 04 05:45:00.179883 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.179751 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3552c62c-8a98-4845-9817-16b284233c6f" containerName="payload-processing" Nov 04 05:45:00.179883 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.179763 5258 cpu_manager.go:414] "containerMap: removing container" podUID="c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1" containerName="payload-processing" Nov 04 05:45:00.179883 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.179768 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1" containerName="payload-processing" Nov 04 05:45:00.180213 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.180069 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="c5196a26-04e3-41d2-ab3a-0e8e5dbd01f1" containerName="payload-processing" Nov 04 05:45:00.180213 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.180100 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="37be0c15-5de0-4b8f-aaa3-5bbe4c186f49" containerName="payload-pre-processing" Nov 04 05:45:00.180213 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.180108 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="3552c62c-8a98-4845-9817-16b284233c6f" containerName="payload-processing" Nov 04 05:45:00.183324 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.183305 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb"] Nov 04 05:45:00.183467 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.183439 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4" Nov 04 05:45:00.186940 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.186917 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb" Nov 04 05:45:00.247452 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.247373 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4"] Nov 04 05:45:00.250013 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.249983 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb"] Nov 04 05:45:00.300851 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.300816 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 05:45:00.301025 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.300854 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 05:45:00.312878 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.312838 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpb54\" (UniqueName: \"kubernetes.io/projected/5a258f0e-0620-4944-bebd-3802097df915-kube-api-access-bpb54\") pod \"maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb\" (UID: \"5a258f0e-0620-4944-bebd-3802097df915\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb" Nov 04 05:45:00.313078 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.312938 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a258f0e-0620-4944-bebd-3802097df915-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb\" (UID: \"5a258f0e-0620-4944-bebd-3802097df915\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb" Nov 04 05:45:00.313078 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.313012 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tzq9\" (UniqueName: \"kubernetes.io/projected/dbdb0216-c592-4801-a52a-41c51ba709b6-kube-api-access-9tzq9\") pod \"maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4\" (UID: \"dbdb0216-c592-4801-a52a-41c51ba709b6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4" Nov 04 05:45:00.313078 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.313059 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbdb0216-c592-4801-a52a-41c51ba709b6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4\" (UID: \"dbdb0216-c592-4801-a52a-41c51ba709b6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4" Nov 04 05:45:00.314590 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.314456 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dnddh\"" Nov 04 05:45:00.414492 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.414452 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tzq9\" (UniqueName: \"kubernetes.io/projected/dbdb0216-c592-4801-a52a-41c51ba709b6-kube-api-access-9tzq9\") pod \"maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4\" (UID: \"dbdb0216-c592-4801-a52a-41c51ba709b6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4" Nov 04 05:45:00.414676 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.414517 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbdb0216-c592-4801-a52a-41c51ba709b6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4\" (UID: \"dbdb0216-c592-4801-a52a-41c51ba709b6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4" Nov 04 05:45:00.414676 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.414601 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpb54\" (UniqueName: \"kubernetes.io/projected/5a258f0e-0620-4944-bebd-3802097df915-kube-api-access-bpb54\") pod \"maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb\" (UID: \"5a258f0e-0620-4944-bebd-3802097df915\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb" Nov 04 05:45:00.414676 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.414663 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a258f0e-0620-4944-bebd-3802097df915-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb\" (UID: \"5a258f0e-0620-4944-bebd-3802097df915\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb" Nov 04 05:45:00.415229 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.415209 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbdb0216-c592-4801-a52a-41c51ba709b6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4\" (UID: \"dbdb0216-c592-4801-a52a-41c51ba709b6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4" Nov 04 05:45:00.415319 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.415300 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a258f0e-0620-4944-bebd-3802097df915-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb\" (UID: \"5a258f0e-0620-4944-bebd-3802097df915\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb" Nov 04 05:45:00.482092 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.482009 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpb54\" (UniqueName: \"kubernetes.io/projected/5a258f0e-0620-4944-bebd-3802097df915-kube-api-access-bpb54\") pod \"maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb\" (UID: \"5a258f0e-0620-4944-bebd-3802097df915\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb" Nov 04 05:45:00.482092 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.482024 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tzq9\" (UniqueName: \"kubernetes.io/projected/dbdb0216-c592-4801-a52a-41c51ba709b6-kube-api-access-9tzq9\") pod \"maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4\" (UID: \"dbdb0216-c592-4801-a52a-41c51ba709b6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4" Nov 04 05:45:00.497883 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.497859 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4" Nov 04 05:45:00.504071 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.504053 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb" Nov 04 05:45:00.968956 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:00.968915 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4"] Nov 04 05:45:00.973503 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:45:00.973467 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbdb0216_c592_4801_a52a_41c51ba709b6.slice/crio-f62c71e86a30f414af5f4481c8eff6245e338fb129e4eb7899cb7a7e5a488417 WatchSource:0}: Error finding container f62c71e86a30f414af5f4481c8eff6245e338fb129e4eb7899cb7a7e5a488417: Status 404 returned error can't find the container with id f62c71e86a30f414af5f4481c8eff6245e338fb129e4eb7899cb7a7e5a488417 Nov 04 05:45:01.023018 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:01.022986 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb"] Nov 04 05:45:01.025912 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:45:01.025886 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a258f0e_0620_4944_bebd_3802097df915.slice/crio-15ab8c164552eca85afcc42e56d59801e5ef4054662bf319cbe9e69451f91183 WatchSource:0}: Error finding container 15ab8c164552eca85afcc42e56d59801e5ef4054662bf319cbe9e69451f91183: Status 404 returned error can't find the container with id 15ab8c164552eca85afcc42e56d59801e5ef4054662bf319cbe9e69451f91183 Nov 04 05:45:01.678791 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:01.678749 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb"] Nov 04 05:45:01.679295 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:01.679088 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" podUID="8e00e063-7fc1-4104-b3b5-4f41f8926b5b" containerName="istio-proxy" containerID="cri-o://0b13eee03f12889e72563f790ab4add2f48b52e67e9aa329a79fe3e0a0b55e2e" gracePeriod=30 Nov 04 05:45:01.748978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:01.748941 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb" event={"ID":"5a258f0e-0620-4944-bebd-3802097df915","Type":"ContainerStarted","Data":"7e2d9e1990789b9926db6c0879405f6e49b31bd6d4984fbc54d2c87bace84264"} Nov 04 05:45:01.749174 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:01.749004 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb" event={"ID":"5a258f0e-0620-4944-bebd-3802097df915","Type":"ContainerStarted","Data":"15ab8c164552eca85afcc42e56d59801e5ef4054662bf319cbe9e69451f91183"} Nov 04 05:45:01.751166 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:01.751014 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4" event={"ID":"dbdb0216-c592-4801-a52a-41c51ba709b6","Type":"ContainerStarted","Data":"afd63140b5a8168836f2ab5c9d8a583b92e7e46eb2c4cec8ede1fd02c3d71247"} Nov 04 05:45:01.751314 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:01.751196 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4" event={"ID":"dbdb0216-c592-4801-a52a-41c51ba709b6","Type":"ContainerStarted","Data":"f62c71e86a30f414af5f4481c8eff6245e338fb129e4eb7899cb7a7e5a488417"} Nov 04 05:45:01.753045 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:01.753007 5258 generic.go:356] "Generic (PLEG): container finished" podID="63d9aa0a-c01b-46e5-b029-278cbeb206d6" containerID="3b3a48ec342a44582544378c9830a0af3121aec789e6be19bbc94eaa5059f991" exitCode=0 Nov 04 05:45:01.753142 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:01.753077 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2" event={"ID":"63d9aa0a-c01b-46e5-b029-278cbeb206d6","Type":"ContainerDied","Data":"3b3a48ec342a44582544378c9830a0af3121aec789e6be19bbc94eaa5059f991"} Nov 04 05:45:01.840258 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:01.840233 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2" Nov 04 05:45:01.931169 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:01.931132 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63d9aa0a-c01b-46e5-b029-278cbeb206d6-kube-api-access-5wnfw\" (UniqueName: \"kubernetes.io/projected/63d9aa0a-c01b-46e5-b029-278cbeb206d6-kube-api-access-5wnfw\") pod \"63d9aa0a-c01b-46e5-b029-278cbeb206d6\" (UID: \"63d9aa0a-c01b-46e5-b029-278cbeb206d6\") " Nov 04 05:45:01.931345 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:01.931187 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63d9aa0a-c01b-46e5-b029-278cbeb206d6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63d9aa0a-c01b-46e5-b029-278cbeb206d6-plugins-config-volume\") pod \"63d9aa0a-c01b-46e5-b029-278cbeb206d6\" (UID: \"63d9aa0a-c01b-46e5-b029-278cbeb206d6\") " Nov 04 05:45:01.931573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:01.931545 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63d9aa0a-c01b-46e5-b029-278cbeb206d6-plugins-config-volume" pod "63d9aa0a-c01b-46e5-b029-278cbeb206d6" (UID: "63d9aa0a-c01b-46e5-b029-278cbeb206d6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:01.933295 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:01.933270 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63d9aa0a-c01b-46e5-b029-278cbeb206d6-kube-api-access-5wnfw" pod "63d9aa0a-c01b-46e5-b029-278cbeb206d6" (UID: "63d9aa0a-c01b-46e5-b029-278cbeb206d6"). InnerVolumeSpecName "kube-api-access-5wnfw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:01.955405 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:01.955355 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4" podStartSLOduration=1.9553412589999999 podStartE2EDuration="1.955341259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:45:01.953468992 +0000 UTC m=+1035.832861464" watchObservedRunningTime="2026-11-04 05:45:01.955341259 +0000 UTC m=+1035.834733731" Nov 04 05:45:02.032455 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:02.032420 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wnfw\" (UniqueName: \"kubernetes.io/projected/63d9aa0a-c01b-46e5-b029-278cbeb206d6-kube-api-access-5wnfw\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:02.032455 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:02.032450 5258 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63d9aa0a-c01b-46e5-b029-278cbeb206d6-plugins-config-volume\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:02.083435 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:02.083321 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb" podStartSLOduration=2.083306158 podStartE2EDuration="2.083306158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:45:02.080947965 +0000 UTC m=+1035.960340438" watchObservedRunningTime="2026-11-04 05:45:02.083306158 +0000 UTC m=+1035.962698631" Nov 04 05:45:02.394410 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:02.394369 5258 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt" podUID="775cd728-84a4-43a3-b3a1-2fdcbb19203e" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 05:45:02.760841 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:02.760761 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2" Nov 04 05:45:02.760841 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:02.760797 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2" event={"ID":"63d9aa0a-c01b-46e5-b029-278cbeb206d6","Type":"ContainerDied","Data":"e44b4c81bd5da65f700a31f1dbde87944f3f9a110e405f339cd238f9c73f4c20"} Nov 04 05:45:02.761384 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:02.760859 5258 scope.go:122] "RemoveContainer" containerID="3b3a48ec342a44582544378c9830a0af3121aec789e6be19bbc94eaa5059f991" Nov 04 05:45:02.763113 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:02.763084 5258 generic.go:356] "Generic (PLEG): container finished" podID="5a258f0e-0620-4944-bebd-3802097df915" containerID="7e2d9e1990789b9926db6c0879405f6e49b31bd6d4984fbc54d2c87bace84264" exitCode=60 Nov 04 05:45:02.763307 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:02.763253 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb" event={"ID":"5a258f0e-0620-4944-bebd-3802097df915","Type":"ContainerDied","Data":"7e2d9e1990789b9926db6c0879405f6e49b31bd6d4984fbc54d2c87bace84264"} Nov 04 05:45:02.763481 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:02.763465 5258 scope.go:122] "RemoveContainer" containerID="7e2d9e1990789b9926db6c0879405f6e49b31bd6d4984fbc54d2c87bace84264" Nov 04 05:45:02.765634 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:02.765610 5258 generic.go:356] "Generic (PLEG): container finished" podID="dbdb0216-c592-4801-a52a-41c51ba709b6" containerID="afd63140b5a8168836f2ab5c9d8a583b92e7e46eb2c4cec8ede1fd02c3d71247" exitCode=60 Nov 04 05:45:02.765743 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:02.765684 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4" event={"ID":"dbdb0216-c592-4801-a52a-41c51ba709b6","Type":"ContainerDied","Data":"afd63140b5a8168836f2ab5c9d8a583b92e7e46eb2c4cec8ede1fd02c3d71247"} Nov 04 05:45:02.765939 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:02.765924 5258 scope.go:122] "RemoveContainer" containerID="afd63140b5a8168836f2ab5c9d8a583b92e7e46eb2c4cec8ede1fd02c3d71247" Nov 04 05:45:03.075061 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:03.075016 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2"] Nov 04 05:45:03.079663 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:03.079636 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-283815-79598958bb-xdnd2"] Nov 04 05:45:03.627581 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:03.627549 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd"] Nov 04 05:45:03.627987 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:03.627957 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" podUID="5cfab36f-9d70-4efa-96ed-a0db577b9939" containerName="istio-proxy" containerID="cri-o://33c053e899edc2496d9e1efe111467119cefe6f08d1e22dd4fe61837357e4c43" gracePeriod=30 Nov 04 05:45:03.774936 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:03.774838 5258 generic.go:356] "Generic (PLEG): container finished" podID="dbdb0216-c592-4801-a52a-41c51ba709b6" containerID="c8df611c16da85f76cf084e924e3fb21ba521393f7e13251b15eb64d1a110a46" exitCode=60 Nov 04 05:45:03.774936 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:03.774919 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4" event={"ID":"dbdb0216-c592-4801-a52a-41c51ba709b6","Type":"ContainerDied","Data":"c8df611c16da85f76cf084e924e3fb21ba521393f7e13251b15eb64d1a110a46"} Nov 04 05:45:03.775493 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:03.774976 5258 scope.go:122] "RemoveContainer" containerID="afd63140b5a8168836f2ab5c9d8a583b92e7e46eb2c4cec8ede1fd02c3d71247" Nov 04 05:45:03.775493 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:03.775286 5258 scope.go:122] "RemoveContainer" containerID="c8df611c16da85f76cf084e924e3fb21ba521393f7e13251b15eb64d1a110a46" Nov 04 05:45:03.775625 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:45:03.775604 5258 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4_odh-ai-gateway-infra(dbdb0216-c592-4801-a52a-41c51ba709b6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4" podUID="dbdb0216-c592-4801-a52a-41c51ba709b6" Nov 04 05:45:03.778193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:03.778166 5258 generic.go:356] "Generic (PLEG): container finished" podID="5a258f0e-0620-4944-bebd-3802097df915" containerID="6cced02d626e18bd0c8d7c04f8d1af3847962d766d462959a89900c5f16f6581" exitCode=60 Nov 04 05:45:03.778300 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:03.778249 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb" event={"ID":"5a258f0e-0620-4944-bebd-3802097df915","Type":"ContainerDied","Data":"6cced02d626e18bd0c8d7c04f8d1af3847962d766d462959a89900c5f16f6581"} Nov 04 05:45:03.778534 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:03.778518 5258 scope.go:122] "RemoveContainer" containerID="6cced02d626e18bd0c8d7c04f8d1af3847962d766d462959a89900c5f16f6581" Nov 04 05:45:03.778809 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:45:03.778785 5258 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb_odh-ai-gateway-infra(5a258f0e-0620-4944-bebd-3802097df915)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb" podUID="5a258f0e-0620-4944-bebd-3802097df915" Nov 04 05:45:03.787049 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:03.787014 5258 scope.go:122] "RemoveContainer" containerID="7e2d9e1990789b9926db6c0879405f6e49b31bd6d4984fbc54d2c87bace84264" Nov 04 05:45:04.269611 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:04.269578 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63d9aa0a-c01b-46e5-b029-278cbeb206d6" path="/var/lib/kubelet/pods/63d9aa0a-c01b-46e5-b029-278cbeb206d6/volumes" Nov 04 05:45:04.787192 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:04.787160 5258 scope.go:122] "RemoveContainer" containerID="c8df611c16da85f76cf084e924e3fb21ba521393f7e13251b15eb64d1a110a46" Nov 04 05:45:04.787682 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:45:04.787491 5258 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4_odh-ai-gateway-infra(dbdb0216-c592-4801-a52a-41c51ba709b6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4" podUID="dbdb0216-c592-4801-a52a-41c51ba709b6" Nov 04 05:45:04.789278 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:04.789255 5258 scope.go:122] "RemoveContainer" containerID="6cced02d626e18bd0c8d7c04f8d1af3847962d766d462959a89900c5f16f6581" Nov 04 05:45:04.789497 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:45:04.789480 5258 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb_odh-ai-gateway-infra(5a258f0e-0620-4944-bebd-3802097df915)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb" podUID="5a258f0e-0620-4944-bebd-3802097df915" Nov 04 05:45:05.154016 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:05.153979 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4"] Nov 04 05:45:05.154302 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:05.154271 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" podUID="86f487ad-d8ba-40ca-9f08-b4f560862516" containerName="istio-proxy" containerID="cri-o://49f1161a112cdd60e00c9717668533bc4bfe6d5059d1a52a243a65f9f4a3e4ee" gracePeriod=30 Nov 04 05:45:06.661538 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:06.661464 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt" Nov 04 05:45:06.781400 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:06.781366 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/775cd728-84a4-43a3-b3a1-2fdcbb19203e-kube-api-access-m72bm\" (UniqueName: \"kubernetes.io/projected/775cd728-84a4-43a3-b3a1-2fdcbb19203e-kube-api-access-m72bm\") pod \"775cd728-84a4-43a3-b3a1-2fdcbb19203e\" (UID: \"775cd728-84a4-43a3-b3a1-2fdcbb19203e\") " Nov 04 05:45:06.781584 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:06.781570 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/775cd728-84a4-43a3-b3a1-2fdcbb19203e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/775cd728-84a4-43a3-b3a1-2fdcbb19203e-plugins-config-volume\") pod \"775cd728-84a4-43a3-b3a1-2fdcbb19203e\" (UID: \"775cd728-84a4-43a3-b3a1-2fdcbb19203e\") " Nov 04 05:45:06.781993 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:06.781963 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/775cd728-84a4-43a3-b3a1-2fdcbb19203e-plugins-config-volume" pod "775cd728-84a4-43a3-b3a1-2fdcbb19203e" (UID: "775cd728-84a4-43a3-b3a1-2fdcbb19203e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:06.783512 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:06.783487 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/775cd728-84a4-43a3-b3a1-2fdcbb19203e-kube-api-access-m72bm" pod "775cd728-84a4-43a3-b3a1-2fdcbb19203e" (UID: "775cd728-84a4-43a3-b3a1-2fdcbb19203e"). InnerVolumeSpecName "kube-api-access-m72bm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:06.802994 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:06.802964 5258 generic.go:356] "Generic (PLEG): container finished" podID="775cd728-84a4-43a3-b3a1-2fdcbb19203e" containerID="1f86f6ed357fdfcdfc354387501d04f8982eef96b928ba591fc7234b01f42d0b" exitCode=0 Nov 04 05:45:06.803149 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:06.803019 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt" Nov 04 05:45:06.803149 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:06.803075 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt" event={"ID":"775cd728-84a4-43a3-b3a1-2fdcbb19203e","Type":"ContainerDied","Data":"1f86f6ed357fdfcdfc354387501d04f8982eef96b928ba591fc7234b01f42d0b"} Nov 04 05:45:06.803149 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:06.803138 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt" event={"ID":"775cd728-84a4-43a3-b3a1-2fdcbb19203e","Type":"ContainerDied","Data":"196c03d2338ed97f9fbc029cada170040ed78c93e2b2d9939e5e6f1c8b271666"} Nov 04 05:45:06.803298 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:06.803173 5258 scope.go:122] "RemoveContainer" containerID="1f86f6ed357fdfcdfc354387501d04f8982eef96b928ba591fc7234b01f42d0b" Nov 04 05:45:06.805164 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:06.805140 5258 generic.go:356] "Generic (PLEG): container finished" podID="8e00e063-7fc1-4104-b3b5-4f41f8926b5b" containerID="0b13eee03f12889e72563f790ab4add2f48b52e67e9aa329a79fe3e0a0b55e2e" exitCode=0 Nov 04 05:45:06.805276 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:06.805200 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" event={"ID":"8e00e063-7fc1-4104-b3b5-4f41f8926b5b","Type":"ContainerDied","Data":"0b13eee03f12889e72563f790ab4add2f48b52e67e9aa329a79fe3e0a0b55e2e"} Nov 04 05:45:06.815756 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:06.815736 5258 scope.go:122] "RemoveContainer" containerID="1f86f6ed357fdfcdfc354387501d04f8982eef96b928ba591fc7234b01f42d0b" Nov 04 05:45:06.816116 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:45:06.816095 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f86f6ed357fdfcdfc354387501d04f8982eef96b928ba591fc7234b01f42d0b\": container with ID starting with 1f86f6ed357fdfcdfc354387501d04f8982eef96b928ba591fc7234b01f42d0b not found: ID does not exist" containerID="1f86f6ed357fdfcdfc354387501d04f8982eef96b928ba591fc7234b01f42d0b" Nov 04 05:45:06.816202 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:06.816129 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f86f6ed357fdfcdfc354387501d04f8982eef96b928ba591fc7234b01f42d0b"} err="failed to get container status \"1f86f6ed357fdfcdfc354387501d04f8982eef96b928ba591fc7234b01f42d0b\": rpc error: code = NotFound desc = could not find container \"1f86f6ed357fdfcdfc354387501d04f8982eef96b928ba591fc7234b01f42d0b\": container with ID starting with 1f86f6ed357fdfcdfc354387501d04f8982eef96b928ba591fc7234b01f42d0b not found: ID does not exist" Nov 04 05:45:06.882474 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:06.882443 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m72bm\" (UniqueName: \"kubernetes.io/projected/775cd728-84a4-43a3-b3a1-2fdcbb19203e-kube-api-access-m72bm\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:06.882474 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:06.882472 5258 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/775cd728-84a4-43a3-b3a1-2fdcbb19203e-plugins-config-volume\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:06.935364 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:06.935338 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:45:06.988566 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:06.988529 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt"] Nov 04 05:45:06.992605 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:06.992577 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-90dc8d-7db59b567f-5dkzt"] Nov 04 05:45:07.084295 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.084255 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-envoy\") pod \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " Nov 04 05:45:07.084485 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.084332 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-workload-socket\") pod \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " Nov 04 05:45:07.084485 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.084391 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-token\" (UniqueName: \"kubernetes.io/projected/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-token\") pod \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " Nov 04 05:45:07.084485 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.084421 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-workload-certs\") pod \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " Nov 04 05:45:07.084485 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.084461 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-credential-socket\") pod \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " Nov 04 05:45:07.084711 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.084508 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-data\") pod \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " Nov 04 05:45:07.084711 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.084572 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-podinfo\") pod \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " Nov 04 05:45:07.084711 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.084609 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-kube-api-access-hcnph\" (UniqueName: \"kubernetes.io/projected/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-kube-api-access-hcnph\") pod \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " Nov 04 05:45:07.084711 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.084639 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-workload-socket" pod "8e00e063-7fc1-4104-b3b5-4f41f8926b5b" (UID: "8e00e063-7fc1-4104-b3b5-4f41f8926b5b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:07.084711 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.084649 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istiod-ca-cert\") pod \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\" (UID: \"8e00e063-7fc1-4104-b3b5-4f41f8926b5b\") " Nov 04 05:45:07.084949 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.084759 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-workload-certs" pod "8e00e063-7fc1-4104-b3b5-4f41f8926b5b" (UID: "8e00e063-7fc1-4104-b3b5-4f41f8926b5b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:07.084949 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.084772 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-credential-socket" pod "8e00e063-7fc1-4104-b3b5-4f41f8926b5b" (UID: "8e00e063-7fc1-4104-b3b5-4f41f8926b5b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:07.085071 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.084997 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istiod-ca-cert" pod "8e00e063-7fc1-4104-b3b5-4f41f8926b5b" (UID: "8e00e063-7fc1-4104-b3b5-4f41f8926b5b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:07.085140 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.085108 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-data" pod "8e00e063-7fc1-4104-b3b5-4f41f8926b5b" (UID: "8e00e063-7fc1-4104-b3b5-4f41f8926b5b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:07.085241 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.085225 5258 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-workload-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:07.085303 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.085243 5258 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-workload-certs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:07.085303 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.085257 5258 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-credential-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:07.085303 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.085269 5258 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-data\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:07.085303 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.085282 5258 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istiod-ca-cert\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:07.086655 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.086631 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-envoy" pod "8e00e063-7fc1-4104-b3b5-4f41f8926b5b" (UID: "8e00e063-7fc1-4104-b3b5-4f41f8926b5b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:07.086796 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.086780 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-kube-api-access-hcnph" pod "8e00e063-7fc1-4104-b3b5-4f41f8926b5b" (UID: "8e00e063-7fc1-4104-b3b5-4f41f8926b5b"). InnerVolumeSpecName "kube-api-access-hcnph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:07.086892 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.086868 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-token" pod "8e00e063-7fc1-4104-b3b5-4f41f8926b5b" (UID: "8e00e063-7fc1-4104-b3b5-4f41f8926b5b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:07.086990 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.086965 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-podinfo" pod "8e00e063-7fc1-4104-b3b5-4f41f8926b5b" (UID: "8e00e063-7fc1-4104-b3b5-4f41f8926b5b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:45:07.187134 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.187057 5258 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-token\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:07.187134 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.187098 5258 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-podinfo\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:07.187134 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.187117 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcnph\" (UniqueName: \"kubernetes.io/projected/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-kube-api-access-hcnph\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:07.187338 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.187134 5258 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e00e063-7fc1-4104-b3b5-4f41f8926b5b-istio-envoy\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:07.812004 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.811972 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" Nov 04 05:45:07.812473 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.812002 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb" event={"ID":"8e00e063-7fc1-4104-b3b5-4f41f8926b5b","Type":"ContainerDied","Data":"502e7acfc3ffe3052b3060a2532618fcf1c7e0d9723ce481bb80812ce6217cf4"} Nov 04 05:45:07.812473 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:07.812079 5258 scope.go:122] "RemoveContainer" containerID="0b13eee03f12889e72563f790ab4add2f48b52e67e9aa329a79fe3e0a0b55e2e" Nov 04 05:45:08.055132 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:08.055102 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb"] Nov 04 05:45:08.058244 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:08.058224 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-90dc8d-openshift-default-6fb649d46f-6rckb"] Nov 04 05:45:08.270119 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:08.270091 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="775cd728-84a4-43a3-b3a1-2fdcbb19203e" path="/var/lib/kubelet/pods/775cd728-84a4-43a3-b3a1-2fdcbb19203e/volumes" Nov 04 05:45:08.270581 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:08.270567 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e00e063-7fc1-4104-b3b5-4f41f8926b5b" path="/var/lib/kubelet/pods/8e00e063-7fc1-4104-b3b5-4f41f8926b5b/volumes" Nov 04 05:45:08.740093 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:08.740057 5258 patch_prober.go:29] interesting pod/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 05:45:08.740250 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:08.740135 5258 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" podUID="86f487ad-d8ba-40ca-9f08-b4f560862516" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 05:45:08.833766 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:08.833713 5258 generic.go:356] "Generic (PLEG): container finished" podID="5cfab36f-9d70-4efa-96ed-a0db577b9939" containerID="33c053e899edc2496d9e1efe111467119cefe6f08d1e22dd4fe61837357e4c43" exitCode=0 Nov 04 05:45:08.834490 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:08.833788 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" event={"ID":"5cfab36f-9d70-4efa-96ed-a0db577b9939","Type":"ContainerDied","Data":"33c053e899edc2496d9e1efe111467119cefe6f08d1e22dd4fe61837357e4c43"} Nov 04 05:45:08.834943 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:08.834923 5258 scope.go:122] "RemoveContainer" containerID="1b0184779bba0f5c00fe4c33fc7bb8c68edb0ad56d17c138302fd4282a345935" Nov 04 05:45:08.906609 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:08.906548 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:45:09.007585 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.007543 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5cfab36f-9d70-4efa-96ed-a0db577b9939-kube-api-access-6nhfs\" (UniqueName: \"kubernetes.io/projected/5cfab36f-9d70-4efa-96ed-a0db577b9939-kube-api-access-6nhfs\") pod \"5cfab36f-9d70-4efa-96ed-a0db577b9939\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " Nov 04 05:45:09.007585 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.007591 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5cfab36f-9d70-4efa-96ed-a0db577b9939-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cfab36f-9d70-4efa-96ed-a0db577b9939-istiod-ca-cert\") pod \"5cfab36f-9d70-4efa-96ed-a0db577b9939\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " Nov 04 05:45:09.007778 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.007646 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-podinfo\") pod \"5cfab36f-9d70-4efa-96ed-a0db577b9939\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " Nov 04 05:45:09.007836 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.007779 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-workload-certs\") pod \"5cfab36f-9d70-4efa-96ed-a0db577b9939\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " Nov 04 05:45:09.007872 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.007858 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-token\" (UniqueName: \"kubernetes.io/projected/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-token\") pod \"5cfab36f-9d70-4efa-96ed-a0db577b9939\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " Nov 04 05:45:09.007908 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.007896 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-envoy\") pod \"5cfab36f-9d70-4efa-96ed-a0db577b9939\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " Nov 04 05:45:09.007954 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.007942 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-workload-socket\") pod \"5cfab36f-9d70-4efa-96ed-a0db577b9939\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " Nov 04 05:45:09.008007 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.007984 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-data\") pod \"5cfab36f-9d70-4efa-96ed-a0db577b9939\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " Nov 04 05:45:09.008110 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.008008 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-workload-certs" pod "5cfab36f-9d70-4efa-96ed-a0db577b9939" (UID: "5cfab36f-9d70-4efa-96ed-a0db577b9939"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:09.008110 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.008061 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cfab36f-9d70-4efa-96ed-a0db577b9939-istiod-ca-cert" pod "5cfab36f-9d70-4efa-96ed-a0db577b9939" (UID: "5cfab36f-9d70-4efa-96ed-a0db577b9939"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:09.008110 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.008075 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-credential-socket\") pod \"5cfab36f-9d70-4efa-96ed-a0db577b9939\" (UID: \"5cfab36f-9d70-4efa-96ed-a0db577b9939\") " Nov 04 05:45:09.008266 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.008190 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-workload-socket" pod "5cfab36f-9d70-4efa-96ed-a0db577b9939" (UID: "5cfab36f-9d70-4efa-96ed-a0db577b9939"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:09.008323 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.008290 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-data" pod "5cfab36f-9d70-4efa-96ed-a0db577b9939" (UID: "5cfab36f-9d70-4efa-96ed-a0db577b9939"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:09.008394 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.008368 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-credential-socket" pod "5cfab36f-9d70-4efa-96ed-a0db577b9939" (UID: "5cfab36f-9d70-4efa-96ed-a0db577b9939"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:09.008602 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.008573 5258 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-workload-certs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:09.008602 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.008605 5258 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-workload-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:09.008785 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.008620 5258 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-data\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:09.008785 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.008648 5258 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-credential-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:09.008785 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.008664 5258 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cfab36f-9d70-4efa-96ed-a0db577b9939-istiod-ca-cert\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:09.009999 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.009973 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-podinfo" pod "5cfab36f-9d70-4efa-96ed-a0db577b9939" (UID: "5cfab36f-9d70-4efa-96ed-a0db577b9939"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:45:09.010155 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.010124 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-envoy" pod "5cfab36f-9d70-4efa-96ed-a0db577b9939" (UID: "5cfab36f-9d70-4efa-96ed-a0db577b9939"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:09.010155 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.010144 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-token" pod "5cfab36f-9d70-4efa-96ed-a0db577b9939" (UID: "5cfab36f-9d70-4efa-96ed-a0db577b9939"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:09.010272 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.010224 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cfab36f-9d70-4efa-96ed-a0db577b9939-kube-api-access-6nhfs" pod "5cfab36f-9d70-4efa-96ed-a0db577b9939" (UID: "5cfab36f-9d70-4efa-96ed-a0db577b9939"). InnerVolumeSpecName "kube-api-access-6nhfs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:09.109808 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.109770 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nhfs\" (UniqueName: \"kubernetes.io/projected/5cfab36f-9d70-4efa-96ed-a0db577b9939-kube-api-access-6nhfs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:09.109808 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.109801 5258 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-podinfo\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:09.109808 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.109811 5258 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-token\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:09.110065 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.109820 5258 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cfab36f-9d70-4efa-96ed-a0db577b9939-istio-envoy\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:09.844412 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.844385 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" Nov 04 05:45:09.844412 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.844384 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd" event={"ID":"5cfab36f-9d70-4efa-96ed-a0db577b9939","Type":"ContainerDied","Data":"1861dd910c25c39c3630ced8311e40566680a6e825329fb22f42a2989c2f50c7"} Nov 04 05:45:09.844939 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:09.844457 5258 scope.go:122] "RemoveContainer" containerID="33c053e899edc2496d9e1efe111467119cefe6f08d1e22dd4fe61837357e4c43" Nov 04 05:45:10.029106 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.029069 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd"] Nov 04 05:45:10.085443 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.085403 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-283815-openshift-default-578d454689-qw4rd"] Nov 04 05:45:10.269743 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.269708 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5cfab36f-9d70-4efa-96ed-a0db577b9939" path="/var/lib/kubelet/pods/5cfab36f-9d70-4efa-96ed-a0db577b9939/volumes" Nov 04 05:45:10.411107 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.411083 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:45:10.523590 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.523504 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/86f487ad-d8ba-40ca-9f08-b4f560862516-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86f487ad-d8ba-40ca-9f08-b4f560862516-istiod-ca-cert\") pod \"86f487ad-d8ba-40ca-9f08-b4f560862516\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " Nov 04 05:45:10.523590 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.523577 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-credential-socket\") pod \"86f487ad-d8ba-40ca-9f08-b4f560862516\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " Nov 04 05:45:10.523774 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.523681 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-podinfo\") pod \"86f487ad-d8ba-40ca-9f08-b4f560862516\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " Nov 04 05:45:10.523774 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.523740 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-workload-socket\") pod \"86f487ad-d8ba-40ca-9f08-b4f560862516\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " Nov 04 05:45:10.523850 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.523815 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-workload-certs\") pod \"86f487ad-d8ba-40ca-9f08-b4f560862516\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " Nov 04 05:45:10.523889 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.523877 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-token\" (UniqueName: \"kubernetes.io/projected/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-token\") pod \"86f487ad-d8ba-40ca-9f08-b4f560862516\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " Nov 04 05:45:10.523938 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.523899 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-credential-socket" pod "86f487ad-d8ba-40ca-9f08-b4f560862516" (UID: "86f487ad-d8ba-40ca-9f08-b4f560862516"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:10.523938 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.523926 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/86f487ad-d8ba-40ca-9f08-b4f560862516-kube-api-access-qnrlh\" (UniqueName: \"kubernetes.io/projected/86f487ad-d8ba-40ca-9f08-b4f560862516-kube-api-access-qnrlh\") pod \"86f487ad-d8ba-40ca-9f08-b4f560862516\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " Nov 04 05:45:10.524071 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.523957 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-workload-socket" pod "86f487ad-d8ba-40ca-9f08-b4f560862516" (UID: "86f487ad-d8ba-40ca-9f08-b4f560862516"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:10.524071 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.523968 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-data\") pod \"86f487ad-d8ba-40ca-9f08-b4f560862516\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " Nov 04 05:45:10.524071 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.524007 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-envoy\") pod \"86f487ad-d8ba-40ca-9f08-b4f560862516\" (UID: \"86f487ad-d8ba-40ca-9f08-b4f560862516\") " Nov 04 05:45:10.524071 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.523968 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86f487ad-d8ba-40ca-9f08-b4f560862516-istiod-ca-cert" pod "86f487ad-d8ba-40ca-9f08-b4f560862516" (UID: "86f487ad-d8ba-40ca-9f08-b4f560862516"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:10.524278 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.524152 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-workload-certs" pod "86f487ad-d8ba-40ca-9f08-b4f560862516" (UID: "86f487ad-d8ba-40ca-9f08-b4f560862516"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:10.524364 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.524347 5258 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86f487ad-d8ba-40ca-9f08-b4f560862516-istiod-ca-cert\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:10.524414 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.524366 5258 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-credential-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:10.524414 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.524376 5258 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-workload-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:10.524414 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.524389 5258 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-workload-certs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:10.524571 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.524480 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-data" pod "86f487ad-d8ba-40ca-9f08-b4f560862516" (UID: "86f487ad-d8ba-40ca-9f08-b4f560862516"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:10.525940 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.525912 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-podinfo" pod "86f487ad-d8ba-40ca-9f08-b4f560862516" (UID: "86f487ad-d8ba-40ca-9f08-b4f560862516"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:45:10.526128 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.526110 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-token" pod "86f487ad-d8ba-40ca-9f08-b4f560862516" (UID: "86f487ad-d8ba-40ca-9f08-b4f560862516"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:10.526203 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.526110 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-envoy" pod "86f487ad-d8ba-40ca-9f08-b4f560862516" (UID: "86f487ad-d8ba-40ca-9f08-b4f560862516"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:10.526203 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.526175 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86f487ad-d8ba-40ca-9f08-b4f560862516-kube-api-access-qnrlh" pod "86f487ad-d8ba-40ca-9f08-b4f560862516" (UID: "86f487ad-d8ba-40ca-9f08-b4f560862516"). InnerVolumeSpecName "kube-api-access-qnrlh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:10.625598 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.625568 5258 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-token\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:10.625598 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.625597 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnrlh\" (UniqueName: \"kubernetes.io/projected/86f487ad-d8ba-40ca-9f08-b4f560862516-kube-api-access-qnrlh\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:10.625598 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.625608 5258 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-data\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:10.625821 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.625617 5258 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-envoy\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:10.625821 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.625625 5258 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86f487ad-d8ba-40ca-9f08-b4f560862516-istio-podinfo\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:10.851889 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.851799 5258 generic.go:356] "Generic (PLEG): container finished" podID="86f487ad-d8ba-40ca-9f08-b4f560862516" containerID="49f1161a112cdd60e00c9717668533bc4bfe6d5059d1a52a243a65f9f4a3e4ee" exitCode=0 Nov 04 05:45:10.851889 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.851844 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" event={"ID":"86f487ad-d8ba-40ca-9f08-b4f560862516","Type":"ContainerDied","Data":"49f1161a112cdd60e00c9717668533bc4bfe6d5059d1a52a243a65f9f4a3e4ee"} Nov 04 05:45:10.851889 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.851847 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" Nov 04 05:45:10.851889 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.851885 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4" event={"ID":"86f487ad-d8ba-40ca-9f08-b4f560862516","Type":"ContainerDied","Data":"0ab9455c82362cdebf55165c1407500c8c33bf4499f0526314653e3daeab7026"} Nov 04 05:45:10.852514 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.851906 5258 scope.go:122] "RemoveContainer" containerID="49f1161a112cdd60e00c9717668533bc4bfe6d5059d1a52a243a65f9f4a3e4ee" Nov 04 05:45:10.861708 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.861686 5258 scope.go:122] "RemoveContainer" containerID="8fc01b5d1790f5726d1ba0e0eece5ac03a9f09e1c4bbea055951536672444cee" Nov 04 05:45:10.869750 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.869734 5258 scope.go:122] "RemoveContainer" containerID="49f1161a112cdd60e00c9717668533bc4bfe6d5059d1a52a243a65f9f4a3e4ee" Nov 04 05:45:10.870002 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:45:10.869984 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49f1161a112cdd60e00c9717668533bc4bfe6d5059d1a52a243a65f9f4a3e4ee\": container with ID starting with 49f1161a112cdd60e00c9717668533bc4bfe6d5059d1a52a243a65f9f4a3e4ee not found: ID does not exist" containerID="49f1161a112cdd60e00c9717668533bc4bfe6d5059d1a52a243a65f9f4a3e4ee" Nov 04 05:45:10.870097 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.870009 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49f1161a112cdd60e00c9717668533bc4bfe6d5059d1a52a243a65f9f4a3e4ee"} err="failed to get container status \"49f1161a112cdd60e00c9717668533bc4bfe6d5059d1a52a243a65f9f4a3e4ee\": rpc error: code = NotFound desc = could not find container \"49f1161a112cdd60e00c9717668533bc4bfe6d5059d1a52a243a65f9f4a3e4ee\": container with ID starting with 49f1161a112cdd60e00c9717668533bc4bfe6d5059d1a52a243a65f9f4a3e4ee not found: ID does not exist" Nov 04 05:45:10.870097 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.870027 5258 scope.go:122] "RemoveContainer" containerID="8fc01b5d1790f5726d1ba0e0eece5ac03a9f09e1c4bbea055951536672444cee" Nov 04 05:45:10.870348 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:45:10.870331 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fc01b5d1790f5726d1ba0e0eece5ac03a9f09e1c4bbea055951536672444cee\": container with ID starting with 8fc01b5d1790f5726d1ba0e0eece5ac03a9f09e1c4bbea055951536672444cee not found: ID does not exist" containerID="8fc01b5d1790f5726d1ba0e0eece5ac03a9f09e1c4bbea055951536672444cee" Nov 04 05:45:10.870400 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:10.870352 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fc01b5d1790f5726d1ba0e0eece5ac03a9f09e1c4bbea055951536672444cee"} err="failed to get container status \"8fc01b5d1790f5726d1ba0e0eece5ac03a9f09e1c4bbea055951536672444cee\": rpc error: code = NotFound desc = could not find container \"8fc01b5d1790f5726d1ba0e0eece5ac03a9f09e1c4bbea055951536672444cee\": container with ID starting with 8fc01b5d1790f5726d1ba0e0eece5ac03a9f09e1c4bbea055951536672444cee not found: ID does not exist" Nov 04 05:45:11.035423 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.035390 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4"] Nov 04 05:45:11.039320 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.039292 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-fbbad9-openshift-default-59cf5b7f65-rwph4"] Nov 04 05:45:11.253856 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.253822 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25"] Nov 04 05:45:11.254267 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.254254 5258 cpu_manager.go:414] "containerMap: removing container" podUID="63d9aa0a-c01b-46e5-b029-278cbeb206d6" containerName="payload-processing" Nov 04 05:45:11.254322 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.254269 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63d9aa0a-c01b-46e5-b029-278cbeb206d6" containerName="payload-processing" Nov 04 05:45:11.254322 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.254279 5258 cpu_manager.go:414] "containerMap: removing container" podUID="775cd728-84a4-43a3-b3a1-2fdcbb19203e" containerName="payload-pre-processing" Nov 04 05:45:11.254322 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.254284 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="775cd728-84a4-43a3-b3a1-2fdcbb19203e" containerName="payload-pre-processing" Nov 04 05:45:11.254322 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.254308 5258 cpu_manager.go:414] "containerMap: removing container" podUID="86f487ad-d8ba-40ca-9f08-b4f560862516" containerName="istio-proxy" Nov 04 05:45:11.254322 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.254313 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="86f487ad-d8ba-40ca-9f08-b4f560862516" containerName="istio-proxy" Nov 04 05:45:11.254322 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.254321 5258 cpu_manager.go:414] "containerMap: removing container" podUID="86f487ad-d8ba-40ca-9f08-b4f560862516" containerName="istio-proxy" Nov 04 05:45:11.254507 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.254326 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="86f487ad-d8ba-40ca-9f08-b4f560862516" containerName="istio-proxy" Nov 04 05:45:11.254507 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.254340 5258 cpu_manager.go:414] "containerMap: removing container" podUID="5cfab36f-9d70-4efa-96ed-a0db577b9939" containerName="istio-proxy" Nov 04 05:45:11.254507 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.254344 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5cfab36f-9d70-4efa-96ed-a0db577b9939" containerName="istio-proxy" Nov 04 05:45:11.254507 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.254354 5258 cpu_manager.go:414] "containerMap: removing container" podUID="5cfab36f-9d70-4efa-96ed-a0db577b9939" containerName="istio-proxy" Nov 04 05:45:11.254507 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.254358 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5cfab36f-9d70-4efa-96ed-a0db577b9939" containerName="istio-proxy" Nov 04 05:45:11.254507 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.254367 5258 cpu_manager.go:414] "containerMap: removing container" podUID="8e00e063-7fc1-4104-b3b5-4f41f8926b5b" containerName="istio-proxy" Nov 04 05:45:11.254507 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.254372 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e00e063-7fc1-4104-b3b5-4f41f8926b5b" containerName="istio-proxy" Nov 04 05:45:11.254507 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.254444 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="86f487ad-d8ba-40ca-9f08-b4f560862516" containerName="istio-proxy" Nov 04 05:45:11.254507 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.254451 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="86f487ad-d8ba-40ca-9f08-b4f560862516" containerName="istio-proxy" Nov 04 05:45:11.254507 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.254456 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="5cfab36f-9d70-4efa-96ed-a0db577b9939" containerName="istio-proxy" Nov 04 05:45:11.254507 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.254465 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="5cfab36f-9d70-4efa-96ed-a0db577b9939" containerName="istio-proxy" Nov 04 05:45:11.254507 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.254470 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e00e063-7fc1-4104-b3b5-4f41f8926b5b" containerName="istio-proxy" Nov 04 05:45:11.254507 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.254477 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="63d9aa0a-c01b-46e5-b029-278cbeb206d6" containerName="payload-processing" Nov 04 05:45:11.254507 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.254484 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="775cd728-84a4-43a3-b3a1-2fdcbb19203e" containerName="payload-pre-processing" Nov 04 05:45:11.259353 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.259332 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.318438 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.318408 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-901a00-openshift-default-dockercfg-txfc4\"" Nov 04 05:45:11.324995 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.324969 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25"] Nov 04 05:45:11.432330 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.432297 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.432479 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.432356 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.432517 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.432491 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.432581 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.432564 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.432638 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.432602 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33978c82-e069-4b5c-bbe9-049f35fb31af-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.432672 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.432653 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.432748 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.432734 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.432785 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.432774 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4852\" (UniqueName: \"kubernetes.io/projected/33978c82-e069-4b5c-bbe9-049f35fb31af-kube-api-access-n4852\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.432843 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.432831 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.534324 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.534232 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.534466 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.534394 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.534466 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.534453 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4852\" (UniqueName: \"kubernetes.io/projected/33978c82-e069-4b5c-bbe9-049f35fb31af-kube-api-access-n4852\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.534557 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.534485 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.534557 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.534509 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.534557 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.534550 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.534701 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.534631 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.534701 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.534680 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.534805 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.534727 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.534805 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.534757 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.534805 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.534763 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33978c82-e069-4b5c-bbe9-049f35fb31af-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.534947 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.534810 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.535047 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.535001 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.535384 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.535367 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33978c82-e069-4b5c-bbe9-049f35fb31af-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.537414 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.537391 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.537627 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.537607 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.601835 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.601800 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.602132 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.602110 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4852\" (UniqueName: \"kubernetes.io/projected/33978c82-e069-4b5c-bbe9-049f35fb31af-kube-api-access-n4852\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:11.872389 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:11.872361 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:12.272007 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:12.271901 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="86f487ad-d8ba-40ca-9f08-b4f560862516" path="/var/lib/kubelet/pods/86f487ad-d8ba-40ca-9f08-b4f560862516/volumes" Nov 04 05:45:12.319401 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:12.319371 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25"] Nov 04 05:45:12.322617 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:45:12.322591 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33978c82_e069_4b5c_bbe9_049f35fb31af.slice/crio-30a379ba89ec571de3c9c273cf3474663a65820fe3f5815476f294cd65601a58 WatchSource:0}: Error finding container 30a379ba89ec571de3c9c273cf3474663a65820fe3f5815476f294cd65601a58: Status 404 returned error can't find the container with id 30a379ba89ec571de3c9c273cf3474663a65820fe3f5815476f294cd65601a58 Nov 04 05:45:12.324798 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:12.324765 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:12.324897 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:12.324842 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:12.324963 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:12.324923 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:12.865054 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:12.864996 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" event={"ID":"33978c82-e069-4b5c-bbe9-049f35fb31af","Type":"ContainerStarted","Data":"068c7091d0485eb0bf3a36ef1bc1d6e482ada7457f0b624fdbc2598717b28cb6"} Nov 04 05:45:12.865237 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:12.865066 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" event={"ID":"33978c82-e069-4b5c-bbe9-049f35fb31af","Type":"ContainerStarted","Data":"30a379ba89ec571de3c9c273cf3474663a65820fe3f5815476f294cd65601a58"} Nov 04 05:45:12.873325 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:12.873297 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:13.874885 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:13.874849 5258 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:45:13.875307 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:13.874899 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" podUID="33978c82-e069-4b5c-bbe9-049f35fb31af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Nov 04 05:45:14.873058 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:14.872988 5258 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:45:14.873223 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:14.873081 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" podUID="33978c82-e069-4b5c-bbe9-049f35fb31af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Nov 04 05:45:15.872744 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:15.872707 5258 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:45:15.873160 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:15.872764 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" podUID="33978c82-e069-4b5c-bbe9-049f35fb31af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Nov 04 05:45:16.268099 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:16.268002 5258 scope.go:122] "RemoveContainer" containerID="c8df611c16da85f76cf084e924e3fb21ba521393f7e13251b15eb64d1a110a46" Nov 04 05:45:16.873345 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:16.873310 5258 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:45:16.873765 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:16.873370 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" podUID="33978c82-e069-4b5c-bbe9-049f35fb31af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Nov 04 05:45:16.889833 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:16.889799 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4" event={"ID":"dbdb0216-c592-4801-a52a-41c51ba709b6","Type":"ContainerStarted","Data":"7dd302b257c07a723587e1e91ede009536a908204f20bb46c0ff5e305a0d4469"} Nov 04 05:45:17.029189 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:17.029087 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" podStartSLOduration=6.029072219 podStartE2EDuration="6.029072219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:45:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:45:13.05843488 +0000 UTC m=+1046.937827354" watchObservedRunningTime="2026-11-04 05:45:17.029072219 +0000 UTC m=+1050.908464683" Nov 04 05:45:17.872836 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:17.872805 5258 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:45:17.873001 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:17.872869 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" podUID="33978c82-e069-4b5c-bbe9-049f35fb31af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Nov 04 05:45:17.896871 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:17.896831 5258 generic.go:356] "Generic (PLEG): container finished" podID="dbdb0216-c592-4801-a52a-41c51ba709b6" containerID="7dd302b257c07a723587e1e91ede009536a908204f20bb46c0ff5e305a0d4469" exitCode=60 Nov 04 05:45:17.897355 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:17.896904 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4" event={"ID":"dbdb0216-c592-4801-a52a-41c51ba709b6","Type":"ContainerDied","Data":"7dd302b257c07a723587e1e91ede009536a908204f20bb46c0ff5e305a0d4469"} Nov 04 05:45:17.897355 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:17.896974 5258 scope.go:122] "RemoveContainer" containerID="c8df611c16da85f76cf084e924e3fb21ba521393f7e13251b15eb64d1a110a46" Nov 04 05:45:17.897464 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:17.897376 5258 scope.go:122] "RemoveContainer" containerID="7dd302b257c07a723587e1e91ede009536a908204f20bb46c0ff5e305a0d4469" Nov 04 05:45:17.897685 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:45:17.897662 5258 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4_odh-ai-gateway-infra(dbdb0216-c592-4801-a52a-41c51ba709b6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4" podUID="dbdb0216-c592-4801-a52a-41c51ba709b6" Nov 04 05:45:18.061218 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:18.061178 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4"] Nov 04 05:45:18.264159 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:18.264079 5258 scope.go:122] "RemoveContainer" containerID="6cced02d626e18bd0c8d7c04f8d1af3847962d766d462959a89900c5f16f6581" Nov 04 05:45:18.873198 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:18.873154 5258 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body= Nov 04 05:45:18.873352 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:18.873225 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" podUID="33978c82-e069-4b5c-bbe9-049f35fb31af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" Nov 04 05:45:18.906370 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:18.906334 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb" event={"ID":"5a258f0e-0620-4944-bebd-3802097df915","Type":"ContainerStarted","Data":"695b03a0a7d32a697ff066d959c5680b9a8927279944936c971bc704b273b9a5"} Nov 04 05:45:19.137292 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:19.137228 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4" Nov 04 05:45:19.222674 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:19.222643 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dbdb0216-c592-4801-a52a-41c51ba709b6-kube-api-access-9tzq9\" (UniqueName: \"kubernetes.io/projected/dbdb0216-c592-4801-a52a-41c51ba709b6-kube-api-access-9tzq9\") pod \"dbdb0216-c592-4801-a52a-41c51ba709b6\" (UID: \"dbdb0216-c592-4801-a52a-41c51ba709b6\") " Nov 04 05:45:19.222831 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:19.222694 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dbdb0216-c592-4801-a52a-41c51ba709b6-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbdb0216-c592-4801-a52a-41c51ba709b6-maas-api-service-ca\") pod \"dbdb0216-c592-4801-a52a-41c51ba709b6\" (UID: \"dbdb0216-c592-4801-a52a-41c51ba709b6\") " Nov 04 05:45:19.223191 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:19.223159 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbdb0216-c592-4801-a52a-41c51ba709b6-maas-api-service-ca" pod "dbdb0216-c592-4801-a52a-41c51ba709b6" (UID: "dbdb0216-c592-4801-a52a-41c51ba709b6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:19.224802 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:19.224770 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbdb0216-c592-4801-a52a-41c51ba709b6-kube-api-access-9tzq9" pod "dbdb0216-c592-4801-a52a-41c51ba709b6" (UID: "dbdb0216-c592-4801-a52a-41c51ba709b6"). InnerVolumeSpecName "kube-api-access-9tzq9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:19.323884 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:19.323845 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tzq9\" (UniqueName: \"kubernetes.io/projected/dbdb0216-c592-4801-a52a-41c51ba709b6-kube-api-access-9tzq9\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:19.323884 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:19.323886 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbdb0216-c592-4801-a52a-41c51ba709b6-maas-api-service-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:19.877306 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:19.877275 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:19.877555 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:19.877536 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:19.878350 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:19.878331 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:45:19.913976 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:19.913955 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4" Nov 04 05:45:19.914418 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:19.913950 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4" event={"ID":"dbdb0216-c592-4801-a52a-41c51ba709b6","Type":"ContainerDied","Data":"f62c71e86a30f414af5f4481c8eff6245e338fb129e4eb7899cb7a7e5a488417"} Nov 04 05:45:19.914418 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:19.914093 5258 scope.go:122] "RemoveContainer" containerID="7dd302b257c07a723587e1e91ede009536a908204f20bb46c0ff5e305a0d4469" Nov 04 05:45:19.916203 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:19.916172 5258 generic.go:356] "Generic (PLEG): container finished" podID="5a258f0e-0620-4944-bebd-3802097df915" containerID="695b03a0a7d32a697ff066d959c5680b9a8927279944936c971bc704b273b9a5" exitCode=60 Nov 04 05:45:19.916299 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:19.916229 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb" event={"ID":"5a258f0e-0620-4944-bebd-3802097df915","Type":"ContainerDied","Data":"695b03a0a7d32a697ff066d959c5680b9a8927279944936c971bc704b273b9a5"} Nov 04 05:45:19.916709 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:19.916690 5258 scope.go:122] "RemoveContainer" containerID="695b03a0a7d32a697ff066d959c5680b9a8927279944936c971bc704b273b9a5" Nov 04 05:45:19.916991 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:45:19.916965 5258 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb_odh-ai-gateway-infra(5a258f0e-0620-4944-bebd-3802097df915)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb" podUID="5a258f0e-0620-4944-bebd-3802097df915" Nov 04 05:45:19.923880 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:19.923864 5258 scope.go:122] "RemoveContainer" containerID="6cced02d626e18bd0c8d7c04f8d1af3847962d766d462959a89900c5f16f6581" Nov 04 05:45:20.134073 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:20.133978 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4"] Nov 04 05:45:20.137639 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:20.137609 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w3-a84d48-29896185-5r4x4"] Nov 04 05:45:20.170502 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:20.170471 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb"] Nov 04 05:45:20.270166 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:20.270139 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dbdb0216-c592-4801-a52a-41c51ba709b6" path="/var/lib/kubelet/pods/dbdb0216-c592-4801-a52a-41c51ba709b6/volumes" Nov 04 05:45:21.071278 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:21.071254 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb" Nov 04 05:45:21.140446 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:21.140415 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5a258f0e-0620-4944-bebd-3802097df915-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a258f0e-0620-4944-bebd-3802097df915-maas-api-service-ca\") pod \"5a258f0e-0620-4944-bebd-3802097df915\" (UID: \"5a258f0e-0620-4944-bebd-3802097df915\") " Nov 04 05:45:21.140608 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:21.140514 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a258f0e-0620-4944-bebd-3802097df915-kube-api-access-bpb54\" (UniqueName: \"kubernetes.io/projected/5a258f0e-0620-4944-bebd-3802097df915-kube-api-access-bpb54\") pod \"5a258f0e-0620-4944-bebd-3802097df915\" (UID: \"5a258f0e-0620-4944-bebd-3802097df915\") " Nov 04 05:45:21.140807 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:21.140785 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a258f0e-0620-4944-bebd-3802097df915-maas-api-service-ca" pod "5a258f0e-0620-4944-bebd-3802097df915" (UID: "5a258f0e-0620-4944-bebd-3802097df915"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:21.142685 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:21.142666 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a258f0e-0620-4944-bebd-3802097df915-kube-api-access-bpb54" pod "5a258f0e-0620-4944-bebd-3802097df915" (UID: "5a258f0e-0620-4944-bebd-3802097df915"). InnerVolumeSpecName "kube-api-access-bpb54". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:21.241535 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:21.241456 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpb54\" (UniqueName: \"kubernetes.io/projected/5a258f0e-0620-4944-bebd-3802097df915-kube-api-access-bpb54\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:21.241535 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:21.241485 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a258f0e-0620-4944-bebd-3802097df915-maas-api-service-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:21.932011 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:21.931980 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb" Nov 04 05:45:21.932221 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:21.932018 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb" event={"ID":"5a258f0e-0620-4944-bebd-3802097df915","Type":"ContainerDied","Data":"15ab8c164552eca85afcc42e56d59801e5ef4054662bf319cbe9e69451f91183"} Nov 04 05:45:21.932221 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:21.932099 5258 scope.go:122] "RemoveContainer" containerID="695b03a0a7d32a697ff066d959c5680b9a8927279944936c971bc704b273b9a5" Nov 04 05:45:22.115546 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:22.115512 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb"] Nov 04 05:45:22.119587 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:22.119559 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-901a00-29896185-2hpdb"] Nov 04 05:45:22.269981 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:22.269890 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5a258f0e-0620-4944-bebd-3802097df915" path="/var/lib/kubelet/pods/5a258f0e-0620-4944-bebd-3802097df915/volumes" Nov 04 05:45:22.781604 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:45:22.781570 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:45:26.545696 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.545665 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85bc275b-rxc8q"] Nov 04 05:45:26.546097 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.546079 5258 cpu_manager.go:414] "containerMap: removing container" podUID="5a258f0e-0620-4944-bebd-3802097df915" containerName="cleanup" Nov 04 05:45:26.546097 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.546090 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a258f0e-0620-4944-bebd-3802097df915" containerName="cleanup" Nov 04 05:45:26.546186 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.546102 5258 cpu_manager.go:414] "containerMap: removing container" podUID="dbdb0216-c592-4801-a52a-41c51ba709b6" containerName="cleanup" Nov 04 05:45:26.546186 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.546107 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dbdb0216-c592-4801-a52a-41c51ba709b6" containerName="cleanup" Nov 04 05:45:26.546186 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.546123 5258 cpu_manager.go:414] "containerMap: removing container" podUID="dbdb0216-c592-4801-a52a-41c51ba709b6" containerName="cleanup" Nov 04 05:45:26.546186 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.546128 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dbdb0216-c592-4801-a52a-41c51ba709b6" containerName="cleanup" Nov 04 05:45:26.546186 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.546138 5258 cpu_manager.go:414] "containerMap: removing container" podUID="5a258f0e-0620-4944-bebd-3802097df915" containerName="cleanup" Nov 04 05:45:26.546186 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.546142 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a258f0e-0620-4944-bebd-3802097df915" containerName="cleanup" Nov 04 05:45:26.546348 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.546218 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="dbdb0216-c592-4801-a52a-41c51ba709b6" containerName="cleanup" Nov 04 05:45:26.546348 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.546225 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a258f0e-0620-4944-bebd-3802097df915" containerName="cleanup" Nov 04 05:45:26.546348 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.546233 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="dbdb0216-c592-4801-a52a-41c51ba709b6" containerName="cleanup" Nov 04 05:45:26.546348 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.546243 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a258f0e-0620-4944-bebd-3802097df915" containerName="cleanup" Nov 04 05:45:26.550754 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.550733 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85bc275b-rxc8q" Nov 04 05:45:26.595820 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.595788 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4mt4\" (UniqueName: \"kubernetes.io/projected/3118c893-72c5-49c2-84c2-bd1b80861767-kube-api-access-z4mt4\") pod \"maas-api-revoke-sub-85bc275b-rxc8q\" (UID: \"3118c893-72c5-49c2-84c2-bd1b80861767\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85bc275b-rxc8q" Nov 04 05:45:26.595979 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.595918 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3118c893-72c5-49c2-84c2-bd1b80861767-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85bc275b-rxc8q\" (UID: \"3118c893-72c5-49c2-84c2-bd1b80861767\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85bc275b-rxc8q" Nov 04 05:45:26.596061 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.595982 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3118c893-72c5-49c2-84c2-bd1b80861767-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85bc275b-rxc8q\" (UID: \"3118c893-72c5-49c2-84c2-bd1b80861767\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85bc275b-rxc8q" Nov 04 05:45:26.610420 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.610393 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 05:45:26.666629 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.666597 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Nov 04 05:45:26.666629 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.666605 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 05:45:26.666811 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.666798 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dnddh\"" Nov 04 05:45:26.668843 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.668827 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85bc275b-rxc8q"] Nov 04 05:45:26.697576 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.697544 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3118c893-72c5-49c2-84c2-bd1b80861767-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85bc275b-rxc8q\" (UID: \"3118c893-72c5-49c2-84c2-bd1b80861767\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85bc275b-rxc8q" Nov 04 05:45:26.697753 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.697589 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3118c893-72c5-49c2-84c2-bd1b80861767-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85bc275b-rxc8q\" (UID: \"3118c893-72c5-49c2-84c2-bd1b80861767\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85bc275b-rxc8q" Nov 04 05:45:26.697753 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.697650 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4mt4\" (UniqueName: \"kubernetes.io/projected/3118c893-72c5-49c2-84c2-bd1b80861767-kube-api-access-z4mt4\") pod \"maas-api-revoke-sub-85bc275b-rxc8q\" (UID: \"3118c893-72c5-49c2-84c2-bd1b80861767\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85bc275b-rxc8q" Nov 04 05:45:26.698288 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.698266 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3118c893-72c5-49c2-84c2-bd1b80861767-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-85bc275b-rxc8q\" (UID: \"3118c893-72c5-49c2-84c2-bd1b80861767\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85bc275b-rxc8q" Nov 04 05:45:26.700093 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.700070 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3118c893-72c5-49c2-84c2-bd1b80861767-maas-api-xks-ca\") pod \"maas-api-revoke-sub-85bc275b-rxc8q\" (UID: \"3118c893-72c5-49c2-84c2-bd1b80861767\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85bc275b-rxc8q" Nov 04 05:45:26.765084 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.765051 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4mt4\" (UniqueName: \"kubernetes.io/projected/3118c893-72c5-49c2-84c2-bd1b80861767-kube-api-access-z4mt4\") pod \"maas-api-revoke-sub-85bc275b-rxc8q\" (UID: \"3118c893-72c5-49c2-84c2-bd1b80861767\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-85bc275b-rxc8q" Nov 04 05:45:26.864091 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:26.864061 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85bc275b-rxc8q" Nov 04 05:45:27.251559 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:27.251485 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-85bc275b-rxc8q"] Nov 04 05:45:27.255078 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:45:27.255018 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3118c893_72c5_49c2_84c2_bd1b80861767.slice/crio-d0d1ed04955bbd3842a4defead0acc093a77707d831614e9a326f217f2923fd4 WatchSource:0}: Error finding container d0d1ed04955bbd3842a4defead0acc093a77707d831614e9a326f217f2923fd4: Status 404 returned error can't find the container with id d0d1ed04955bbd3842a4defead0acc093a77707d831614e9a326f217f2923fd4 Nov 04 05:45:27.567190 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:27.567158 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:45:27.965397 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:27.965357 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85bc275b-rxc8q" event={"ID":"3118c893-72c5-49c2-84c2-bd1b80861767","Type":"ContainerStarted","Data":"09cff4fad3dc7d3cba9af50b763d84558baeda99f6aefce9cd057f04fc513289"} Nov 04 05:45:27.965553 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:27.965421 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85bc275b-rxc8q" event={"ID":"3118c893-72c5-49c2-84c2-bd1b80861767","Type":"ContainerStarted","Data":"d0d1ed04955bbd3842a4defead0acc093a77707d831614e9a326f217f2923fd4"} Nov 04 05:45:28.093718 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:28.093654 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85bc275b-rxc8q" podStartSLOduration=2.09363653 podStartE2EDuration="2.09363653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:45:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:45:28.092640702 +0000 UTC m=+1061.972033180" watchObservedRunningTime="2026-11-04 05:45:28.09363653 +0000 UTC m=+1061.973029004" Nov 04 05:45:28.975119 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:28.975075 5258 generic.go:356] "Generic (PLEG): container finished" podID="3118c893-72c5-49c2-84c2-bd1b80861767" containerID="09cff4fad3dc7d3cba9af50b763d84558baeda99f6aefce9cd057f04fc513289" exitCode=0 Nov 04 05:45:28.975606 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:28.975183 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85bc275b-rxc8q" event={"ID":"3118c893-72c5-49c2-84c2-bd1b80861767","Type":"ContainerDied","Data":"09cff4fad3dc7d3cba9af50b763d84558baeda99f6aefce9cd057f04fc513289"} Nov 04 05:45:29.439077 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.439043 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79"] Nov 04 05:45:29.439481 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.439469 5258 cpu_manager.go:414] "containerMap: removing container" podUID="5a258f0e-0620-4944-bebd-3802097df915" containerName="cleanup" Nov 04 05:45:29.439532 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.439482 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a258f0e-0620-4944-bebd-3802097df915" containerName="cleanup" Nov 04 05:45:29.439532 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.439501 5258 cpu_manager.go:414] "containerMap: removing container" podUID="dbdb0216-c592-4801-a52a-41c51ba709b6" containerName="cleanup" Nov 04 05:45:29.439532 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.439507 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dbdb0216-c592-4801-a52a-41c51ba709b6" containerName="cleanup" Nov 04 05:45:29.439641 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.439576 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a258f0e-0620-4944-bebd-3802097df915" containerName="cleanup" Nov 04 05:45:29.439641 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.439587 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="dbdb0216-c592-4801-a52a-41c51ba709b6" containerName="cleanup" Nov 04 05:45:29.442554 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.442538 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:29.501333 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.501301 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-961485\"/\"openshift-service-ca.crt\"" Nov 04 05:45:29.501490 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.501367 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-961485\"/\"kube-root-ca.crt\"" Nov 04 05:45:29.501490 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.501385 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-961485\"/\"default-dockercfg-mzgh6\"" Nov 04 05:45:29.501490 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.501398 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-961485\"/\"distinct-2-w2-961485-kserve-self-signed-certs\"" Nov 04 05:45:29.507026 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.507003 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79"] Nov 04 05:45:29.531060 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.531022 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-dshm\") pod \"distinct-2-w2-961485-kserve-588d499c6f-jdj79\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:29.531060 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.531066 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-home\") pod \"distinct-2-w2-961485-kserve-588d499c6f-jdj79\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:29.531246 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.531146 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wklph\" (UniqueName: \"kubernetes.io/projected/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-kube-api-access-wklph\") pod \"distinct-2-w2-961485-kserve-588d499c6f-jdj79\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:29.531246 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.531230 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-model-cache\") pod \"distinct-2-w2-961485-kserve-588d499c6f-jdj79\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:29.531328 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.531274 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-tmp-dir\") pod \"distinct-2-w2-961485-kserve-588d499c6f-jdj79\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:29.531328 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.531290 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-tls-certs\") pod \"distinct-2-w2-961485-kserve-588d499c6f-jdj79\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:29.632222 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.632190 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wklph\" (UniqueName: \"kubernetes.io/projected/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-kube-api-access-wklph\") pod \"distinct-2-w2-961485-kserve-588d499c6f-jdj79\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:29.632382 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.632258 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-model-cache\") pod \"distinct-2-w2-961485-kserve-588d499c6f-jdj79\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:29.632382 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.632308 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-tmp-dir\") pod \"distinct-2-w2-961485-kserve-588d499c6f-jdj79\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:29.632382 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.632329 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-tls-certs\") pod \"distinct-2-w2-961485-kserve-588d499c6f-jdj79\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:29.632558 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.632396 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-dshm\") pod \"distinct-2-w2-961485-kserve-588d499c6f-jdj79\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:29.632558 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.632418 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-home\") pod \"distinct-2-w2-961485-kserve-588d499c6f-jdj79\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:29.632736 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.632713 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-tmp-dir\") pod \"distinct-2-w2-961485-kserve-588d499c6f-jdj79\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:29.632797 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.632729 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-model-cache\") pod \"distinct-2-w2-961485-kserve-588d499c6f-jdj79\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:29.632797 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.632768 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-home\") pod \"distinct-2-w2-961485-kserve-588d499c6f-jdj79\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:29.634570 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.634552 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-dshm\") pod \"distinct-2-w2-961485-kserve-588d499c6f-jdj79\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:29.634905 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.634888 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-tls-certs\") pod \"distinct-2-w2-961485-kserve-588d499c6f-jdj79\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:29.702320 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.702245 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wklph\" (UniqueName: \"kubernetes.io/projected/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-kube-api-access-wklph\") pod \"distinct-2-w2-961485-kserve-588d499c6f-jdj79\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:29.753870 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:29.753839 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:30.107615 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:30.107591 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85bc275b-rxc8q" Nov 04 05:45:30.139635 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:30.139605 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79"] Nov 04 05:45:30.142706 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:45:30.142677 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod964baa93_a9cb_4d82_9ee8_4ca1d7cfc8fd.slice/crio-6e7919c5782d5360251fac950dab3806e03f3017ddcf49a1c16d81b8b352d999 WatchSource:0}: Error finding container 6e7919c5782d5360251fac950dab3806e03f3017ddcf49a1c16d81b8b352d999: Status 404 returned error can't find the container with id 6e7919c5782d5360251fac950dab3806e03f3017ddcf49a1c16d81b8b352d999 Nov 04 05:45:30.252888 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:30.252811 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3118c893-72c5-49c2-84c2-bd1b80861767-kube-api-access-z4mt4\" (UniqueName: \"kubernetes.io/projected/3118c893-72c5-49c2-84c2-bd1b80861767-kube-api-access-z4mt4\") pod \"3118c893-72c5-49c2-84c2-bd1b80861767\" (UID: \"3118c893-72c5-49c2-84c2-bd1b80861767\") " Nov 04 05:45:30.252888 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:30.252881 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3118c893-72c5-49c2-84c2-bd1b80861767-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3118c893-72c5-49c2-84c2-bd1b80861767-maas-api-xks-ca\") pod \"3118c893-72c5-49c2-84c2-bd1b80861767\" (UID: \"3118c893-72c5-49c2-84c2-bd1b80861767\") " Nov 04 05:45:30.253147 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:30.252938 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3118c893-72c5-49c2-84c2-bd1b80861767-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3118c893-72c5-49c2-84c2-bd1b80861767-maas-api-openshift-ca\") pod \"3118c893-72c5-49c2-84c2-bd1b80861767\" (UID: \"3118c893-72c5-49c2-84c2-bd1b80861767\") " Nov 04 05:45:30.253507 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:30.253466 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3118c893-72c5-49c2-84c2-bd1b80861767-maas-api-openshift-ca" pod "3118c893-72c5-49c2-84c2-bd1b80861767" (UID: "3118c893-72c5-49c2-84c2-bd1b80861767"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:30.255521 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:30.255485 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3118c893-72c5-49c2-84c2-bd1b80861767-maas-api-xks-ca" pod "3118c893-72c5-49c2-84c2-bd1b80861767" (UID: "3118c893-72c5-49c2-84c2-bd1b80861767"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:45:30.255590 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:30.255491 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3118c893-72c5-49c2-84c2-bd1b80861767-kube-api-access-z4mt4" pod "3118c893-72c5-49c2-84c2-bd1b80861767" (UID: "3118c893-72c5-49c2-84c2-bd1b80861767"). InnerVolumeSpecName "kube-api-access-z4mt4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:30.354653 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:30.354619 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4mt4\" (UniqueName: \"kubernetes.io/projected/3118c893-72c5-49c2-84c2-bd1b80861767-kube-api-access-z4mt4\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.354653 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:30.354647 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3118c893-72c5-49c2-84c2-bd1b80861767-maas-api-xks-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.354653 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:30.354656 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3118c893-72c5-49c2-84c2-bd1b80861767-maas-api-openshift-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:30.815505 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:30.815467 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242"] Nov 04 05:45:30.840482 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:30.840454 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n"] Nov 04 05:45:30.988835 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:30.988736 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" event={"ID":"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd","Type":"ContainerStarted","Data":"ee786e026d7432e45c73cd3c38654f93c06d644b4855e062775f67bd3203990a"} Nov 04 05:45:30.988835 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:30.988797 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" event={"ID":"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd","Type":"ContainerStarted","Data":"6e7919c5782d5360251fac950dab3806e03f3017ddcf49a1c16d81b8b352d999"} Nov 04 05:45:30.990072 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:30.990017 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85bc275b-rxc8q" event={"ID":"3118c893-72c5-49c2-84c2-bd1b80861767","Type":"ContainerDied","Data":"d0d1ed04955bbd3842a4defead0acc093a77707d831614e9a326f217f2923fd4"} Nov 04 05:45:30.990072 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:30.990064 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-85bc275b-rxc8q" Nov 04 05:45:30.990225 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:30.990123 5258 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0d1ed04955bbd3842a4defead0acc093a77707d831614e9a326f217f2923fd4" Nov 04 05:45:31.266597 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:31.266495 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" podStartSLOduration=2.26647994 podStartE2EDuration="2.26647994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:45:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:45:31.264882171 +0000 UTC m=+1065.144274640" watchObservedRunningTime="2026-11-04 05:45:31.26647994 +0000 UTC m=+1065.145872412" Nov 04 05:45:31.754902 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:31.754869 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:38.410419 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:38.410383 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-736fd121-nxnn7"] Nov 04 05:45:38.410843 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:38.410820 5258 cpu_manager.go:414] "containerMap: removing container" podUID="3118c893-72c5-49c2-84c2-bd1b80861767" containerName="revoke-keys" Nov 04 05:45:38.410843 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:38.410832 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3118c893-72c5-49c2-84c2-bd1b80861767" containerName="revoke-keys" Nov 04 05:45:38.410926 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:38.410919 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="3118c893-72c5-49c2-84c2-bd1b80861767" containerName="revoke-keys" Nov 04 05:45:38.414677 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:38.414656 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-736fd121-nxnn7" Nov 04 05:45:38.474558 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:38.474527 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 05:45:38.530768 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:38.530728 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 05:45:38.530935 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:38.530783 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dnddh\"" Nov 04 05:45:38.530935 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:38.530817 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Nov 04 05:45:38.533028 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:38.533007 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-736fd121-nxnn7"] Nov 04 05:45:38.536443 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:38.536422 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4e5142d4-4efa-4dcb-bcd6-add8904b076f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-736fd121-nxnn7\" (UID: \"4e5142d4-4efa-4dcb-bcd6-add8904b076f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-736fd121-nxnn7" Nov 04 05:45:38.536537 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:38.536483 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4e5142d4-4efa-4dcb-bcd6-add8904b076f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-736fd121-nxnn7\" (UID: \"4e5142d4-4efa-4dcb-bcd6-add8904b076f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-736fd121-nxnn7" Nov 04 05:45:38.536616 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:38.536599 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnhpd\" (UniqueName: \"kubernetes.io/projected/4e5142d4-4efa-4dcb-bcd6-add8904b076f-kube-api-access-qnhpd\") pod \"maas-api-revoke-sub-736fd121-nxnn7\" (UID: \"4e5142d4-4efa-4dcb-bcd6-add8904b076f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-736fd121-nxnn7" Nov 04 05:45:38.638256 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:38.638205 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4e5142d4-4efa-4dcb-bcd6-add8904b076f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-736fd121-nxnn7\" (UID: \"4e5142d4-4efa-4dcb-bcd6-add8904b076f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-736fd121-nxnn7" Nov 04 05:45:38.638533 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:38.638270 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnhpd\" (UniqueName: \"kubernetes.io/projected/4e5142d4-4efa-4dcb-bcd6-add8904b076f-kube-api-access-qnhpd\") pod \"maas-api-revoke-sub-736fd121-nxnn7\" (UID: \"4e5142d4-4efa-4dcb-bcd6-add8904b076f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-736fd121-nxnn7" Nov 04 05:45:38.638533 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:38.638439 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4e5142d4-4efa-4dcb-bcd6-add8904b076f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-736fd121-nxnn7\" (UID: \"4e5142d4-4efa-4dcb-bcd6-add8904b076f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-736fd121-nxnn7" Nov 04 05:45:38.639110 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:38.639087 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4e5142d4-4efa-4dcb-bcd6-add8904b076f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-736fd121-nxnn7\" (UID: \"4e5142d4-4efa-4dcb-bcd6-add8904b076f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-736fd121-nxnn7" Nov 04 05:45:38.640541 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:38.640518 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4e5142d4-4efa-4dcb-bcd6-add8904b076f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-736fd121-nxnn7\" (UID: \"4e5142d4-4efa-4dcb-bcd6-add8904b076f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-736fd121-nxnn7" Nov 04 05:45:38.705354 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:38.705289 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnhpd\" (UniqueName: \"kubernetes.io/projected/4e5142d4-4efa-4dcb-bcd6-add8904b076f-kube-api-access-qnhpd\") pod \"maas-api-revoke-sub-736fd121-nxnn7\" (UID: \"4e5142d4-4efa-4dcb-bcd6-add8904b076f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-736fd121-nxnn7" Nov 04 05:45:38.727286 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:38.727261 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-736fd121-nxnn7" Nov 04 05:45:39.116191 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:39.116157 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-736fd121-nxnn7"] Nov 04 05:45:39.119882 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:45:39.119844 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e5142d4_4efa_4dcb_bcd6_add8904b076f.slice/crio-233f509d9b861448be6652609c421b559ac6e44b0d5ecb1e174e291202aee5cf WatchSource:0}: Error finding container 233f509d9b861448be6652609c421b559ac6e44b0d5ecb1e174e291202aee5cf: Status 404 returned error can't find the container with id 233f509d9b861448be6652609c421b559ac6e44b0d5ecb1e174e291202aee5cf Nov 04 05:45:39.754351 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:39.754267 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:39.767182 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:39.767157 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:39.799781 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:39.799749 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7"] Nov 04 05:45:39.803559 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:39.803543 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:39.911868 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:39.911831 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7"] Nov 04 05:45:39.953329 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:39.953294 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-token\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:39.953513 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:39.953335 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-envoy\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:39.953513 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:39.953363 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx5lt\" (UniqueName: \"kubernetes.io/projected/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-kube-api-access-dx5lt\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:39.953513 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:39.953432 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-data\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:39.953513 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:39.953498 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-podinfo\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:39.953651 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:39.953540 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-credential-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:39.953651 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:39.953623 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-workload-certs\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:39.953714 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:39.953686 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istiod-ca-cert\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:39.953714 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:39.953703 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-workload-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:40.041950 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.041914 5258 generic.go:356] "Generic (PLEG): container finished" podID="4e5142d4-4efa-4dcb-bcd6-add8904b076f" containerID="d6aa59884b81b6d8507a1114aaa31fdf11e84f2512da0e2f61d93976262cab30" exitCode=0 Nov 04 05:45:40.042144 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.041979 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-736fd121-nxnn7" event={"ID":"4e5142d4-4efa-4dcb-bcd6-add8904b076f","Type":"ContainerDied","Data":"d6aa59884b81b6d8507a1114aaa31fdf11e84f2512da0e2f61d93976262cab30"} Nov 04 05:45:40.042144 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.042023 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-736fd121-nxnn7" event={"ID":"4e5142d4-4efa-4dcb-bcd6-add8904b076f","Type":"ContainerStarted","Data":"233f509d9b861448be6652609c421b559ac6e44b0d5ecb1e174e291202aee5cf"} Nov 04 05:45:40.054013 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.053983 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:45:40.054242 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.054220 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx5lt\" (UniqueName: \"kubernetes.io/projected/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-kube-api-access-dx5lt\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:40.054289 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.054254 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-data\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:40.054289 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.054280 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-podinfo\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:40.054402 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.054328 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-credential-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:40.054402 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.054374 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-workload-certs\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:40.054507 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.054462 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istiod-ca-cert\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:40.054507 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.054486 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-workload-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:40.054629 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.054566 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-token\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:40.054629 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.054601 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-envoy\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:40.054737 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.054674 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-data\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:40.054737 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.054693 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-credential-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:40.054851 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.054798 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-workload-certs\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:40.054964 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.054943 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-workload-socket\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:40.055233 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.055211 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istiod-ca-cert\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:40.057133 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.057104 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-podinfo\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:40.057226 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.057154 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-envoy\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:40.121773 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.121737 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-token\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:40.121932 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.121831 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx5lt\" (UniqueName: \"kubernetes.io/projected/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-kube-api-access-dx5lt\") pod \"e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:40.416861 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.416822 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:40.867437 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.867413 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7"] Nov 04 05:45:40.870318 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:45:40.870285 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1dc03dc_e144_4e2b_9dcb_0fc83a2dc119.slice/crio-c4824ff62121fffb336a1978dcf9cf6e02d0aa2513779b4999459e3fc82df4dd WatchSource:0}: Error finding container c4824ff62121fffb336a1978dcf9cf6e02d0aa2513779b4999459e3fc82df4dd: Status 404 returned error can't find the container with id c4824ff62121fffb336a1978dcf9cf6e02d0aa2513779b4999459e3fc82df4dd Nov 04 05:45:40.872890 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.872854 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:40.872961 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.872923 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:40.872961 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:40.872952 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:41.049715 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:41.049681 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" event={"ID":"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119","Type":"ContainerStarted","Data":"1645dc4b6f665c4b29ef77a54d081df0ed7fc3528ad10966f14439ef107d9cd9"} Nov 04 05:45:41.049909 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:41.049740 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" event={"ID":"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119","Type":"ContainerStarted","Data":"c4824ff62121fffb336a1978dcf9cf6e02d0aa2513779b4999459e3fc82df4dd"} Nov 04 05:45:41.183988 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:41.183935 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" podStartSLOduration=2.183919908 podStartE2EDuration="2.183919908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:45:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:45:41.182299738 +0000 UTC m=+1075.061692213" watchObservedRunningTime="2026-11-04 05:45:41.183919908 +0000 UTC m=+1075.063312468" Nov 04 05:45:41.195065 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:41.195022 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-736fd121-nxnn7" Nov 04 05:45:41.266164 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:41.266126 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4e5142d4-4efa-4dcb-bcd6-add8904b076f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4e5142d4-4efa-4dcb-bcd6-add8904b076f-maas-api-openshift-ca\") pod \"4e5142d4-4efa-4dcb-bcd6-add8904b076f\" (UID: \"4e5142d4-4efa-4dcb-bcd6-add8904b076f\") " Nov 04 05:45:41.266338 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:41.266193 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4e5142d4-4efa-4dcb-bcd6-add8904b076f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4e5142d4-4efa-4dcb-bcd6-add8904b076f-maas-api-xks-ca\") pod \"4e5142d4-4efa-4dcb-bcd6-add8904b076f\" (UID: \"4e5142d4-4efa-4dcb-bcd6-add8904b076f\") " Nov 04 05:45:41.266382 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:41.266336 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e5142d4-4efa-4dcb-bcd6-add8904b076f-kube-api-access-qnhpd\" (UniqueName: \"kubernetes.io/projected/4e5142d4-4efa-4dcb-bcd6-add8904b076f-kube-api-access-qnhpd\") pod \"4e5142d4-4efa-4dcb-bcd6-add8904b076f\" (UID: \"4e5142d4-4efa-4dcb-bcd6-add8904b076f\") " Nov 04 05:45:41.266579 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:41.266552 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e5142d4-4efa-4dcb-bcd6-add8904b076f-maas-api-openshift-ca" pod "4e5142d4-4efa-4dcb-bcd6-add8904b076f" (UID: "4e5142d4-4efa-4dcb-bcd6-add8904b076f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:45:41.268359 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:41.268334 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e5142d4-4efa-4dcb-bcd6-add8904b076f-maas-api-xks-ca" pod "4e5142d4-4efa-4dcb-bcd6-add8904b076f" (UID: "4e5142d4-4efa-4dcb-bcd6-add8904b076f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:45:41.268359 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:41.268335 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e5142d4-4efa-4dcb-bcd6-add8904b076f-kube-api-access-qnhpd" pod "4e5142d4-4efa-4dcb-bcd6-add8904b076f" (UID: "4e5142d4-4efa-4dcb-bcd6-add8904b076f"). InnerVolumeSpecName "kube-api-access-qnhpd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:41.370056 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:41.367786 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnhpd\" (UniqueName: \"kubernetes.io/projected/4e5142d4-4efa-4dcb-bcd6-add8904b076f-kube-api-access-qnhpd\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:41.370056 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:41.367827 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4e5142d4-4efa-4dcb-bcd6-add8904b076f-maas-api-openshift-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:41.370056 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:41.367846 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4e5142d4-4efa-4dcb-bcd6-add8904b076f-maas-api-xks-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:41.417229 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:41.417190 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:41.418939 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:41.418914 5258 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body= Nov 04 05:45:41.419072 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:41.418961 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" podUID="d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" Nov 04 05:45:42.065921 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:42.065827 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-736fd121-nxnn7" Nov 04 05:45:42.065921 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:42.065856 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-736fd121-nxnn7" event={"ID":"4e5142d4-4efa-4dcb-bcd6-add8904b076f","Type":"ContainerDied","Data":"233f509d9b861448be6652609c421b559ac6e44b0d5ecb1e174e291202aee5cf"} Nov 04 05:45:42.065921 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:42.065904 5258 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="233f509d9b861448be6652609c421b559ac6e44b0d5ecb1e174e291202aee5cf" Nov 04 05:45:42.418210 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:42.418171 5258 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body= Nov 04 05:45:42.418366 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:42.418229 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" podUID="d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" Nov 04 05:45:43.417521 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:43.417483 5258 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body= Nov 04 05:45:43.417915 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:43.417536 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" podUID="d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" Nov 04 05:45:44.417795 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:44.417756 5258 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body= Nov 04 05:45:44.418214 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:44.417809 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" podUID="d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" Nov 04 05:45:45.417828 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:45.417793 5258 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body= Nov 04 05:45:45.418238 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:45.417849 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" podUID="d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" Nov 04 05:45:45.858367 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:45.858250 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" podUID="f9f3696d-d689-431f-8da6-a0e2ce5039a1" containerName="main" containerID="cri-o://1b7d4b53437b658700dcaaae12eefc1d5bde06517abce54aef1df50b1b4d6788" gracePeriod=45 Nov 04 05:45:45.860102 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:45.860066 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" podUID="13309444-7174-4fe6-9a47-d26d545a156f" containerName="main" containerID="cri-o://cf1a202f861423c50c4c0bfb6549a8da57bb538d4bbc035d950a5cdc0f5f8207" gracePeriod=45 Nov 04 05:45:45.864601 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:45.864570 5258 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" podUID="13309444-7174-4fe6-9a47-d26d545a156f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.51:8000/ready\": EOF" Nov 04 05:45:46.092252 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.092223 5258 generic.go:356] "Generic (PLEG): container finished" podID="f9f3696d-d689-431f-8da6-a0e2ce5039a1" containerID="1b7d4b53437b658700dcaaae12eefc1d5bde06517abce54aef1df50b1b4d6788" exitCode=0 Nov 04 05:45:46.092391 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.092299 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" event={"ID":"f9f3696d-d689-431f-8da6-a0e2ce5039a1","Type":"ContainerDied","Data":"1b7d4b53437b658700dcaaae12eefc1d5bde06517abce54aef1df50b1b4d6788"} Nov 04 05:45:46.093838 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.093815 5258 generic.go:356] "Generic (PLEG): container finished" podID="13309444-7174-4fe6-9a47-d26d545a156f" containerID="cf1a202f861423c50c4c0bfb6549a8da57bb538d4bbc035d950a5cdc0f5f8207" exitCode=0 Nov 04 05:45:46.093922 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.093887 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" event={"ID":"13309444-7174-4fe6-9a47-d26d545a156f","Type":"ContainerDied","Data":"cf1a202f861423c50c4c0bfb6549a8da57bb538d4bbc035d950a5cdc0f5f8207"} Nov 04 05:45:46.173154 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.173129 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:45:46.187450 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.187429 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:45:46.321070 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.321018 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-dshm\") pod \"13309444-7174-4fe6-9a47-d26d545a156f\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " Nov 04 05:45:46.321327 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.321306 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-home\" (UniqueName: \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-home\") pod \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " Nov 04 05:45:46.321507 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.321471 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9f3696d-d689-431f-8da6-a0e2ce5039a1-kube-api-access-r26dv\" (UniqueName: \"kubernetes.io/projected/f9f3696d-d689-431f-8da6-a0e2ce5039a1-kube-api-access-r26dv\") pod \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " Nov 04 05:45:46.321932 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.321902 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-home" pod "f9f3696d-d689-431f-8da6-a0e2ce5039a1" (UID: "f9f3696d-d689-431f-8da6-a0e2ce5039a1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:46.322114 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.322094 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-tmp-dir\") pod \"13309444-7174-4fe6-9a47-d26d545a156f\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " Nov 04 05:45:46.322330 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.322313 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f9f3696d-d689-431f-8da6-a0e2ce5039a1-tls-certs\" (UniqueName: \"kubernetes.io/secret/f9f3696d-d689-431f-8da6-a0e2ce5039a1-tls-certs\") pod \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " Nov 04 05:45:46.322478 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.322461 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/13309444-7174-4fe6-9a47-d26d545a156f-kube-api-access-49xgj\" (UniqueName: \"kubernetes.io/projected/13309444-7174-4fe6-9a47-d26d545a156f-kube-api-access-49xgj\") pod \"13309444-7174-4fe6-9a47-d26d545a156f\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " Nov 04 05:45:46.322639 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.322624 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-tmp-dir\") pod \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " Nov 04 05:45:46.322759 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.322743 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/13309444-7174-4fe6-9a47-d26d545a156f-tls-certs\" (UniqueName: \"kubernetes.io/secret/13309444-7174-4fe6-9a47-d26d545a156f-tls-certs\") pod \"13309444-7174-4fe6-9a47-d26d545a156f\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " Nov 04 05:45:46.322896 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.322881 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-model-cache\") pod \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " Nov 04 05:45:46.323073 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.323055 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-home\" (UniqueName: \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-home\") pod \"13309444-7174-4fe6-9a47-d26d545a156f\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " Nov 04 05:45:46.323220 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.323204 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-model-cache\") pod \"13309444-7174-4fe6-9a47-d26d545a156f\" (UID: \"13309444-7174-4fe6-9a47-d26d545a156f\") " Nov 04 05:45:46.323488 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.323407 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-dshm\") pod \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\" (UID: \"f9f3696d-d689-431f-8da6-a0e2ce5039a1\") " Nov 04 05:45:46.324166 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.324045 5258 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-home\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.324289 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.322228 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-tmp-dir" pod "13309444-7174-4fe6-9a47-d26d545a156f" (UID: "13309444-7174-4fe6-9a47-d26d545a156f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:46.324382 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.322894 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-tmp-dir" pod "f9f3696d-d689-431f-8da6-a0e2ce5039a1" (UID: "f9f3696d-d689-431f-8da6-a0e2ce5039a1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:46.324469 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.323205 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-model-cache" pod "f9f3696d-d689-431f-8da6-a0e2ce5039a1" (UID: "f9f3696d-d689-431f-8da6-a0e2ce5039a1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:46.324547 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.323382 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-model-cache" pod "13309444-7174-4fe6-9a47-d26d545a156f" (UID: "13309444-7174-4fe6-9a47-d26d545a156f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:46.326284 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.326243 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-home" pod "13309444-7174-4fe6-9a47-d26d545a156f" (UID: "13309444-7174-4fe6-9a47-d26d545a156f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:46.328387 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.328361 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-dshm" pod "13309444-7174-4fe6-9a47-d26d545a156f" (UID: "13309444-7174-4fe6-9a47-d26d545a156f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:46.328587 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.328549 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9f3696d-d689-431f-8da6-a0e2ce5039a1-tls-certs" pod "f9f3696d-d689-431f-8da6-a0e2ce5039a1" (UID: "f9f3696d-d689-431f-8da6-a0e2ce5039a1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:45:46.328587 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.328561 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13309444-7174-4fe6-9a47-d26d545a156f-kube-api-access-49xgj" pod "13309444-7174-4fe6-9a47-d26d545a156f" (UID: "13309444-7174-4fe6-9a47-d26d545a156f"). InnerVolumeSpecName "kube-api-access-49xgj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:46.329224 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.329189 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13309444-7174-4fe6-9a47-d26d545a156f-tls-certs" pod "13309444-7174-4fe6-9a47-d26d545a156f" (UID: "13309444-7174-4fe6-9a47-d26d545a156f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:45:46.330536 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.330508 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-dshm" pod "f9f3696d-d689-431f-8da6-a0e2ce5039a1" (UID: "f9f3696d-d689-431f-8da6-a0e2ce5039a1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:45:46.332428 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.332397 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9f3696d-d689-431f-8da6-a0e2ce5039a1-kube-api-access-r26dv" pod "f9f3696d-d689-431f-8da6-a0e2ce5039a1" (UID: "f9f3696d-d689-431f-8da6-a0e2ce5039a1"). InnerVolumeSpecName "kube-api-access-r26dv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:45:46.418129 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.418097 5258 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body= Nov 04 05:45:46.418498 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.418160 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" podUID="d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" Nov 04 05:45:46.425723 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.425664 5258 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-home\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.425723 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.425694 5258 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-model-cache\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.425723 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.425711 5258 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-dshm\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.425898 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.425724 5258 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-dshm\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.425898 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.425739 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r26dv\" (UniqueName: \"kubernetes.io/projected/f9f3696d-d689-431f-8da6-a0e2ce5039a1-kube-api-access-r26dv\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.425898 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.425753 5258 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/13309444-7174-4fe6-9a47-d26d545a156f-tmp-dir\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.425898 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.425762 5258 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9f3696d-d689-431f-8da6-a0e2ce5039a1-tls-certs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.425898 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.425775 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49xgj\" (UniqueName: \"kubernetes.io/projected/13309444-7174-4fe6-9a47-d26d545a156f-kube-api-access-49xgj\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.425898 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.425786 5258 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-tmp-dir\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.425898 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.425794 5258 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/13309444-7174-4fe6-9a47-d26d545a156f-tls-certs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:46.425898 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:46.425801 5258 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9f3696d-d689-431f-8da6-a0e2ce5039a1-model-cache\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:45:47.105551 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:47.105514 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" Nov 04 05:45:47.105551 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:47.105534 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242" event={"ID":"f9f3696d-d689-431f-8da6-a0e2ce5039a1","Type":"ContainerDied","Data":"9eb9318d8e61ae1d7a676d91e960a34880349a309aee3225fc4a39dd0dea9106"} Nov 04 05:45:47.105849 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:47.105592 5258 scope.go:122] "RemoveContainer" containerID="1b7d4b53437b658700dcaaae12eefc1d5bde06517abce54aef1df50b1b4d6788" Nov 04 05:45:47.107292 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:47.107268 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" Nov 04 05:45:47.107401 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:47.107288 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n" event={"ID":"13309444-7174-4fe6-9a47-d26d545a156f","Type":"ContainerDied","Data":"b0d5b2a0bc6b942a5129d4d0a6933711c5c8266db8989bf447633e5ad69f4de7"} Nov 04 05:45:47.115624 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:47.115599 5258 scope.go:122] "RemoveContainer" containerID="cf1a202f861423c50c4c0bfb6549a8da57bb538d4bbc035d950a5cdc0f5f8207" Nov 04 05:45:47.296935 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:47.296900 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n"] Nov 04 05:45:47.306528 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:47.306498 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/simulator-w3-a84d48-kserve-67675f8597-z9g2n"] Nov 04 05:45:47.417485 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:47.417397 5258 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body= Nov 04 05:45:47.417485 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:47.417459 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" podUID="d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" Nov 04 05:45:47.432649 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:47.432612 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242"] Nov 04 05:45:47.438449 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:47.437135 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-a84d48/distinct-2-w3-a84d48-kserve-667694b496-lw242"] Nov 04 05:45:48.273543 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:48.273508 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="13309444-7174-4fe6-9a47-d26d545a156f" path="/var/lib/kubelet/pods/13309444-7174-4fe6-9a47-d26d545a156f/volumes" Nov 04 05:45:48.274522 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:48.274496 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f9f3696d-d689-431f-8da6-a0e2ce5039a1" path="/var/lib/kubelet/pods/f9f3696d-d689-431f-8da6-a0e2ce5039a1/volumes" Nov 04 05:45:49.459225 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:49.459181 5258 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 05:45:49.459740 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:49.459275 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" podUID="d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 05:45:49.771985 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:49.771885 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:49.772353 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:49.772295 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:49.773118 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:49.773097 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:45:52.780848 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:45:52.780811 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:45:57.004078 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.003982 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c"] Nov 04 05:45:57.004468 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.004429 5258 cpu_manager.go:414] "containerMap: removing container" podUID="13309444-7174-4fe6-9a47-d26d545a156f" containerName="main" Nov 04 05:45:57.004468 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.004441 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="13309444-7174-4fe6-9a47-d26d545a156f" containerName="main" Nov 04 05:45:57.004468 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.004461 5258 cpu_manager.go:414] "containerMap: removing container" podUID="f9f3696d-d689-431f-8da6-a0e2ce5039a1" containerName="main" Nov 04 05:45:57.004468 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.004466 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9f3696d-d689-431f-8da6-a0e2ce5039a1" containerName="main" Nov 04 05:45:57.004594 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.004478 5258 cpu_manager.go:414] "containerMap: removing container" podUID="4e5142d4-4efa-4dcb-bcd6-add8904b076f" containerName="revoke-keys" Nov 04 05:45:57.004594 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.004483 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e5142d4-4efa-4dcb-bcd6-add8904b076f" containerName="revoke-keys" Nov 04 05:45:57.004594 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.004551 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e5142d4-4efa-4dcb-bcd6-add8904b076f" containerName="revoke-keys" Nov 04 05:45:57.004594 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.004563 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="13309444-7174-4fe6-9a47-d26d545a156f" containerName="main" Nov 04 05:45:57.004594 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.004572 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9f3696d-d689-431f-8da6-a0e2ce5039a1" containerName="main" Nov 04 05:45:57.009754 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.009734 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.122102 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.122066 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c"] Nov 04 05:45:57.144610 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.144574 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.144803 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.144643 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rljdf\" (UniqueName: \"kubernetes.io/projected/8aa179d9-60b7-4dec-a891-a379cb56e798-kube-api-access-rljdf\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.144803 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.144734 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8aa179d9-60b7-4dec-a891-a379cb56e798-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.144803 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.144777 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.144980 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.144956 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.145022 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.144987 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.145022 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.145005 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.145165 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.145151 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.145207 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.145176 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.246684 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.246642 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.246684 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.246685 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.246890 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.246762 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.246890 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.246790 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.246890 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.246828 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.246890 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.246876 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rljdf\" (UniqueName: \"kubernetes.io/projected/8aa179d9-60b7-4dec-a891-a379cb56e798-kube-api-access-rljdf\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.247143 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.246903 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8aa179d9-60b7-4dec-a891-a379cb56e798-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.247143 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.246937 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.247143 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.247076 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.247292 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.247222 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.247292 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.247253 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.247292 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.247262 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.247492 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.247469 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.247745 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.247726 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8aa179d9-60b7-4dec-a891-a379cb56e798-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.249050 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.249017 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.249354 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.249335 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.314885 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.314811 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.315029 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.314879 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rljdf\" (UniqueName: \"kubernetes.io/projected/8aa179d9-60b7-4dec-a891-a379cb56e798-kube-api-access-rljdf\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.323581 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.323558 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:57.770284 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.770210 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c"] Nov 04 05:45:57.773166 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:45:57.773135 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aa179d9_60b7_4dec_a891_a379cb56e798.slice/crio-e1261068cfe8e782aed48aae79d334e72dfbe0c4f65598b15500f3105a2832fc WatchSource:0}: Error finding container e1261068cfe8e782aed48aae79d334e72dfbe0c4f65598b15500f3105a2832fc: Status 404 returned error can't find the container with id e1261068cfe8e782aed48aae79d334e72dfbe0c4f65598b15500f3105a2832fc Nov 04 05:45:57.775459 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.775423 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:57.775568 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.775505 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:57.775568 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.775546 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:45:57.903333 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.903298 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz"] Nov 04 05:45:57.907751 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.907729 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:45:57.967749 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.967483 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-901a00\"/\"distinct-2-w4-901a00-kserve-self-signed-certs\"" Nov 04 05:45:57.970056 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:57.970001 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz"] Nov 04 05:45:58.055057 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.054935 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-tmp-dir\") pod \"distinct-2-w4-901a00-kserve-549f59c9df-ktcmz\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:45:58.055695 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.055671 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-model-cache\") pod \"distinct-2-w4-901a00-kserve-549f59c9df-ktcmz\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:45:58.057412 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.057380 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-tls-certs\") pod \"distinct-2-w4-901a00-kserve-549f59c9df-ktcmz\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:45:58.057696 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.057673 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfqjh\" (UniqueName: \"kubernetes.io/projected/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-kube-api-access-bfqjh\") pod \"distinct-2-w4-901a00-kserve-549f59c9df-ktcmz\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:45:58.057868 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.057846 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-home\") pod \"distinct-2-w4-901a00-kserve-549f59c9df-ktcmz\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:45:58.058594 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.058573 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-dshm\") pod \"distinct-2-w4-901a00-kserve-549f59c9df-ktcmz\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:45:58.160003 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.159959 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-home\") pod \"distinct-2-w4-901a00-kserve-549f59c9df-ktcmz\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:45:58.160003 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.160002 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-dshm\") pod \"distinct-2-w4-901a00-kserve-549f59c9df-ktcmz\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:45:58.160268 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.160061 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-tmp-dir\") pod \"distinct-2-w4-901a00-kserve-549f59c9df-ktcmz\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:45:58.160268 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.160120 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-model-cache\") pod \"distinct-2-w4-901a00-kserve-549f59c9df-ktcmz\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:45:58.160268 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.160161 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-tls-certs\") pod \"distinct-2-w4-901a00-kserve-549f59c9df-ktcmz\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:45:58.160268 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.160217 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfqjh\" (UniqueName: \"kubernetes.io/projected/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-kube-api-access-bfqjh\") pod \"distinct-2-w4-901a00-kserve-549f59c9df-ktcmz\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:45:58.160494 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.160408 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-home\") pod \"distinct-2-w4-901a00-kserve-549f59c9df-ktcmz\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:45:58.160494 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.160487 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-tmp-dir\") pod \"distinct-2-w4-901a00-kserve-549f59c9df-ktcmz\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:45:58.160602 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.160558 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-model-cache\") pod \"distinct-2-w4-901a00-kserve-549f59c9df-ktcmz\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:45:58.162313 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.162293 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-dshm\") pod \"distinct-2-w4-901a00-kserve-549f59c9df-ktcmz\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:45:58.162667 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.162643 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-tls-certs\") pod \"distinct-2-w4-901a00-kserve-549f59c9df-ktcmz\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:45:58.169495 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.169461 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" event={"ID":"8aa179d9-60b7-4dec-a891-a379cb56e798","Type":"ContainerStarted","Data":"712dce42fd00f50a14504a08158db950a4f0aa0d4980c805e9cc6e89b2376675"} Nov 04 05:45:58.169622 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.169517 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" event={"ID":"8aa179d9-60b7-4dec-a891-a379cb56e798","Type":"ContainerStarted","Data":"e1261068cfe8e782aed48aae79d334e72dfbe0c4f65598b15500f3105a2832fc"} Nov 04 05:45:58.228300 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.228259 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfqjh\" (UniqueName: \"kubernetes.io/projected/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-kube-api-access-bfqjh\") pod \"distinct-2-w4-901a00-kserve-549f59c9df-ktcmz\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:45:58.310425 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.310310 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" podStartSLOduration=2.310294583 podStartE2EDuration="2.310294583s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:45:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:45:58.307364887 +0000 UTC m=+1092.186757360" watchObservedRunningTime="2026-11-04 05:45:58.310294583 +0000 UTC m=+1092.189687122" Nov 04 05:45:58.324314 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.324281 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:45:58.326231 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.326203 5258 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body= Nov 04 05:45:58.326384 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.326261 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" podUID="8aa179d9-60b7-4dec-a891-a379cb56e798" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" Nov 04 05:45:58.520661 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.520628 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:45:58.953553 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:58.953519 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz"] Nov 04 05:45:58.965557 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:45:58.965520 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e3dfd3b_ab7f_4872_b247_9cfeb0456112.slice/crio-bc504024878d656aa59fe9573930af32728d3d712fc0574c6c02b94b40bbc3a4 WatchSource:0}: Error finding container bc504024878d656aa59fe9573930af32728d3d712fc0574c6c02b94b40bbc3a4: Status 404 returned error can't find the container with id bc504024878d656aa59fe9573930af32728d3d712fc0574c6c02b94b40bbc3a4 Nov 04 05:45:59.178053 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:59.177952 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" event={"ID":"9e3dfd3b-ab7f-4872-b247-9cfeb0456112","Type":"ContainerStarted","Data":"c96230dadb42f30d18cd7bacaec2cedec98c0408610d4e468786c539a1ace0c8"} Nov 04 05:45:59.178053 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:59.178007 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" event={"ID":"9e3dfd3b-ab7f-4872-b247-9cfeb0456112","Type":"ContainerStarted","Data":"bc504024878d656aa59fe9573930af32728d3d712fc0574c6c02b94b40bbc3a4"} Nov 04 05:45:59.317107 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:59.316984 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" podStartSLOduration=2.316965299 podStartE2EDuration="2.316965299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:45:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:45:59.314459677 +0000 UTC m=+1093.193852153" watchObservedRunningTime="2026-11-04 05:45:59.316965299 +0000 UTC m=+1093.196357774" Nov 04 05:45:59.324237 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:59.324207 5258 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body= Nov 04 05:45:59.324365 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:59.324263 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" podUID="8aa179d9-60b7-4dec-a891-a379cb56e798" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" Nov 04 05:45:59.521847 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:45:59.521803 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:46:00.324225 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:00.324190 5258 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body= Nov 04 05:46:00.324621 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:00.324244 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" podUID="8aa179d9-60b7-4dec-a891-a379cb56e798" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" Nov 04 05:46:01.324586 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:01.324547 5258 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body= Nov 04 05:46:01.325004 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:01.324612 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" podUID="8aa179d9-60b7-4dec-a891-a379cb56e798" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" Nov 04 05:46:02.324245 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:02.324214 5258 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body= Nov 04 05:46:02.324412 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:02.324264 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" podUID="8aa179d9-60b7-4dec-a891-a379cb56e798" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" Nov 04 05:46:03.324788 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:03.324753 5258 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body= Nov 04 05:46:03.325222 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:03.324816 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" podUID="8aa179d9-60b7-4dec-a891-a379cb56e798" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" Nov 04 05:46:04.324969 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:04.324927 5258 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body= Nov 04 05:46:04.325497 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:04.324987 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" podUID="8aa179d9-60b7-4dec-a891-a379cb56e798" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" Nov 04 05:46:05.329236 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:05.329204 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:46:05.329743 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:05.329592 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:46:05.330360 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:05.330341 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:46:07.538082 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:07.538045 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:46:08.521140 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:08.521100 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:46:08.534444 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:08.534413 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:46:09.249860 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:09.249822 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:46:17.173271 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:17.173236 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7"] Nov 04 05:46:17.173668 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:17.173486 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" podUID="d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" containerName="istio-proxy" containerID="cri-o://1645dc4b6f665c4b29ef77a54d081df0ed7fc3528ad10966f14439ef107d9cd9" gracePeriod=30 Nov 04 05:46:17.182337 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:17.182309 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm"] Nov 04 05:46:17.182539 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:17.182508 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" podUID="6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" containerName="istio-proxy" containerID="cri-o://a979edbfe06ae280a2f2250734a0f9fcb5faef085aaf00aa6cd8d552f84dd384" gracePeriod=30 Nov 04 05:46:19.775104 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:19.775064 5258 patch_prober.go:29] interesting pod/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 05:46:19.775577 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:19.775132 5258 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" podUID="d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 05:46:22.313069 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.312963 5258 generic.go:356] "Generic (PLEG): container finished" podID="6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" containerID="a979edbfe06ae280a2f2250734a0f9fcb5faef085aaf00aa6cd8d552f84dd384" exitCode=0 Nov 04 05:46:22.313407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.313161 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" event={"ID":"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15","Type":"ContainerDied","Data":"a979edbfe06ae280a2f2250734a0f9fcb5faef085aaf00aa6cd8d552f84dd384"} Nov 04 05:46:22.313407 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.313233 5258 scope.go:122] "RemoveContainer" containerID="79e87858931c3487bd37db62c4fef611d201642dbf9ad792ba351318b73991a8" Nov 04 05:46:22.316277 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.316092 5258 generic.go:356] "Generic (PLEG): container finished" podID="d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" containerID="1645dc4b6f665c4b29ef77a54d081df0ed7fc3528ad10966f14439ef107d9cd9" exitCode=0 Nov 04 05:46:22.316277 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.316196 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" event={"ID":"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119","Type":"ContainerDied","Data":"1645dc4b6f665c4b29ef77a54d081df0ed7fc3528ad10966f14439ef107d9cd9"} Nov 04 05:46:22.441089 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.441011 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:46:22.474117 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.474093 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:46:22.522930 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.522895 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-workload-socket\") pod \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " Nov 04 05:46:22.523141 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.522953 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-podinfo\") pod \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " Nov 04 05:46:22.523141 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.522984 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istiod-ca-cert\") pod \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " Nov 04 05:46:22.523141 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.523010 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-kube-api-access-dx5lt\" (UniqueName: \"kubernetes.io/projected/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-kube-api-access-dx5lt\") pod \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " Nov 04 05:46:22.523141 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.523080 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-credential-socket\") pod \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " Nov 04 05:46:22.523141 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.523115 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-token\" (UniqueName: \"kubernetes.io/projected/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-token\") pod \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " Nov 04 05:46:22.523410 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.523187 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-workload-certs\") pod \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " Nov 04 05:46:22.523410 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.523209 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-envoy\") pod \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " Nov 04 05:46:22.523410 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.523263 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-workload-socket" pod "d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" (UID: "d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.523410 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.523276 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-data\") pod \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\" (UID: \"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119\") " Nov 04 05:46:22.523410 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.523392 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-credential-socket" pod "d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" (UID: "d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.523624 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.523504 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-data" pod "d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" (UID: "d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.523624 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.523539 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istiod-ca-cert" pod "d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" (UID: "d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:22.523624 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.523559 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-workload-certs" pod "d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" (UID: "d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.523799 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.523780 5258 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-workload-certs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.523860 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.523802 5258 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-data\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.523860 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.523814 5258 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-workload-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.523860 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.523832 5258 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istiod-ca-cert\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.523860 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.523851 5258 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-credential-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.525399 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.525361 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-envoy" pod "d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" (UID: "d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.525499 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.525432 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-token" pod "d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" (UID: "d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:22.525499 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.525454 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-podinfo" pod "d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" (UID: "d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:46:22.525499 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.525432 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-kube-api-access-dx5lt" pod "d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" (UID: "d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119"). InnerVolumeSpecName "kube-api-access-dx5lt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:22.624434 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.624401 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-credential-socket\") pod \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " Nov 04 05:46:22.624617 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.624488 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istiod-ca-cert\") pod \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " Nov 04 05:46:22.624617 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.624529 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-kube-api-access-hb4gj\" (UniqueName: \"kubernetes.io/projected/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-kube-api-access-hb4gj\") pod \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " Nov 04 05:46:22.624617 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.624558 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-podinfo\") pod \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " Nov 04 05:46:22.624782 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.624624 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-envoy\") pod \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " Nov 04 05:46:22.624782 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.624699 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-token\" (UniqueName: \"kubernetes.io/projected/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-token\") pod \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " Nov 04 05:46:22.624782 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.624718 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-credential-socket" pod "6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" (UID: "6a3d0b5a-4e73-4939-97f5-f662b5bc6b15"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.624782 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.624752 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-workload-socket\") pod \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " Nov 04 05:46:22.625000 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.624789 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-workload-certs\") pod \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " Nov 04 05:46:22.625000 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.624825 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-data\") pod \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\" (UID: \"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15\") " Nov 04 05:46:22.625158 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.625107 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-workload-socket" pod "6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" (UID: "6a3d0b5a-4e73-4939-97f5-f662b5bc6b15"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.625226 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.625196 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-workload-certs" pod "6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" (UID: "6a3d0b5a-4e73-4939-97f5-f662b5bc6b15"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.625319 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.625300 5258 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-credential-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.625375 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.625327 5258 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-podinfo\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.625375 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.625343 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dx5lt\" (UniqueName: \"kubernetes.io/projected/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-kube-api-access-dx5lt\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.625375 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.625357 5258 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-token\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.625375 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.625371 5258 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-workload-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.625529 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.625385 5258 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-workload-certs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.625529 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.625353 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-data" pod "6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" (UID: "6a3d0b5a-4e73-4939-97f5-f662b5bc6b15"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.625529 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.625399 5258 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119-istio-envoy\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.625529 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.625375 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istiod-ca-cert" pod "6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" (UID: "6a3d0b5a-4e73-4939-97f5-f662b5bc6b15"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:22.626900 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.626871 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-podinfo" pod "6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" (UID: "6a3d0b5a-4e73-4939-97f5-f662b5bc6b15"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:46:22.627256 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.627235 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-envoy" pod "6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" (UID: "6a3d0b5a-4e73-4939-97f5-f662b5bc6b15"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:22.627322 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.627258 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-token" pod "6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" (UID: "6a3d0b5a-4e73-4939-97f5-f662b5bc6b15"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:22.627529 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.627509 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-kube-api-access-hb4gj" pod "6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" (UID: "6a3d0b5a-4e73-4939-97f5-f662b5bc6b15"). InnerVolumeSpecName "kube-api-access-hb4gj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:22.726393 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.726300 5258 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istiod-ca-cert\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.726393 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.726331 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hb4gj\" (UniqueName: \"kubernetes.io/projected/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-kube-api-access-hb4gj\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.726393 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.726343 5258 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-podinfo\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.726393 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.726352 5258 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-envoy\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.726393 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.726361 5258 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-token\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.726393 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:22.726369 5258 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15-istio-data\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:22.784907 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:46:22.784862 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:46:23.322719 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:23.322688 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" Nov 04 05:46:23.323232 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:23.322715 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm" event={"ID":"6a3d0b5a-4e73-4939-97f5-f662b5bc6b15","Type":"ContainerDied","Data":"3bfff75a2389fb89d9813cc6ae58f03c4798532c616c810d91b2dd17b729001d"} Nov 04 05:46:23.323232 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:23.322783 5258 scope.go:122] "RemoveContainer" containerID="a979edbfe06ae280a2f2250734a0f9fcb5faef085aaf00aa6cd8d552f84dd384" Nov 04 05:46:23.324284 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:23.324213 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" Nov 04 05:46:23.324284 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:23.324233 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7" event={"ID":"d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119","Type":"ContainerDied","Data":"c4824ff62121fffb336a1978dcf9cf6e02d0aa2513779b4999459e3fc82df4dd"} Nov 04 05:46:23.332591 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:23.332567 5258 scope.go:122] "RemoveContainer" containerID="1645dc4b6f665c4b29ef77a54d081df0ed7fc3528ad10966f14439ef107d9cd9" Nov 04 05:46:23.507823 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:23.507740 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7"] Nov 04 05:46:23.564167 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:23.564135 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-6mnq7"] Nov 04 05:46:23.686342 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:23.686312 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm"] Nov 04 05:46:23.690254 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:23.690228 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-a84d48-openshift-default-d8668c897-ss6dm"] Nov 04 05:46:24.271005 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:24.270971 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" path="/var/lib/kubelet/pods/6a3d0b5a-4e73-4939-97f5-f662b5bc6b15/volumes" Nov 04 05:46:24.271498 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:24.271482 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" path="/var/lib/kubelet/pods/d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119/volumes" Nov 04 05:46:27.799406 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:27.799369 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:46:28.212701 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.212659 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-df357346-kn6w8"] Nov 04 05:46:28.213179 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.213160 5258 cpu_manager.go:414] "containerMap: removing container" podUID="d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" containerName="istio-proxy" Nov 04 05:46:28.213288 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.213181 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" containerName="istio-proxy" Nov 04 05:46:28.213288 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.213215 5258 cpu_manager.go:414] "containerMap: removing container" podUID="6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" containerName="istio-proxy" Nov 04 05:46:28.213288 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.213223 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" containerName="istio-proxy" Nov 04 05:46:28.213442 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.213360 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1dc03dc-e144-4e2b-9dcb-0fc83a2dc119" containerName="istio-proxy" Nov 04 05:46:28.213442 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.213379 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" containerName="istio-proxy" Nov 04 05:46:28.218367 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.218345 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-df357346-kn6w8" Nov 04 05:46:28.280514 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.280479 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-df357346-kn6w8"] Nov 04 05:46:28.333780 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.333740 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Nov 04 05:46:28.333780 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.333775 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 05:46:28.333978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.333776 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 05:46:28.333978 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.333845 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dnddh\"" Nov 04 05:46:28.384941 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.384910 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/415eeabb-bacf-453b-801c-cf123ac99615-maas-api-xks-ca\") pod \"maas-api-revoke-sub-df357346-kn6w8\" (UID: \"415eeabb-bacf-453b-801c-cf123ac99615\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-df357346-kn6w8" Nov 04 05:46:28.385150 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.385083 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p964\" (UniqueName: \"kubernetes.io/projected/415eeabb-bacf-453b-801c-cf123ac99615-kube-api-access-2p964\") pod \"maas-api-revoke-sub-df357346-kn6w8\" (UID: \"415eeabb-bacf-453b-801c-cf123ac99615\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-df357346-kn6w8" Nov 04 05:46:28.385277 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.385238 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/415eeabb-bacf-453b-801c-cf123ac99615-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-df357346-kn6w8\" (UID: \"415eeabb-bacf-453b-801c-cf123ac99615\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-df357346-kn6w8" Nov 04 05:46:28.486088 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.485983 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p964\" (UniqueName: \"kubernetes.io/projected/415eeabb-bacf-453b-801c-cf123ac99615-kube-api-access-2p964\") pod \"maas-api-revoke-sub-df357346-kn6w8\" (UID: \"415eeabb-bacf-453b-801c-cf123ac99615\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-df357346-kn6w8" Nov 04 05:46:28.486262 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.486104 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/415eeabb-bacf-453b-801c-cf123ac99615-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-df357346-kn6w8\" (UID: \"415eeabb-bacf-453b-801c-cf123ac99615\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-df357346-kn6w8" Nov 04 05:46:28.486262 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.486149 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/415eeabb-bacf-453b-801c-cf123ac99615-maas-api-xks-ca\") pod \"maas-api-revoke-sub-df357346-kn6w8\" (UID: \"415eeabb-bacf-453b-801c-cf123ac99615\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-df357346-kn6w8" Nov 04 05:46:28.486725 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.486705 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/415eeabb-bacf-453b-801c-cf123ac99615-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-df357346-kn6w8\" (UID: \"415eeabb-bacf-453b-801c-cf123ac99615\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-df357346-kn6w8" Nov 04 05:46:28.488530 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.488510 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/415eeabb-bacf-453b-801c-cf123ac99615-maas-api-xks-ca\") pod \"maas-api-revoke-sub-df357346-kn6w8\" (UID: \"415eeabb-bacf-453b-801c-cf123ac99615\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-df357346-kn6w8" Nov 04 05:46:28.553439 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.553411 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p964\" (UniqueName: \"kubernetes.io/projected/415eeabb-bacf-453b-801c-cf123ac99615-kube-api-access-2p964\") pod \"maas-api-revoke-sub-df357346-kn6w8\" (UID: \"415eeabb-bacf-453b-801c-cf123ac99615\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-df357346-kn6w8" Nov 04 05:46:28.830791 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:28.830713 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-df357346-kn6w8" Nov 04 05:46:29.216000 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:29.215917 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-df357346-kn6w8"] Nov 04 05:46:29.218782 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:46:29.218752 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod415eeabb_bacf_453b_801c_cf123ac99615.slice/crio-dfc979588a86a359fdf5edd7cb566b0d36dced51c6435fc1de5b88cca161a59f WatchSource:0}: Error finding container dfc979588a86a359fdf5edd7cb566b0d36dced51c6435fc1de5b88cca161a59f: Status 404 returned error can't find the container with id dfc979588a86a359fdf5edd7cb566b0d36dced51c6435fc1de5b88cca161a59f Nov 04 05:46:29.357964 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:29.357926 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-df357346-kn6w8" event={"ID":"415eeabb-bacf-453b-801c-cf123ac99615","Type":"ContainerStarted","Data":"dfc979588a86a359fdf5edd7cb566b0d36dced51c6435fc1de5b88cca161a59f"} Nov 04 05:46:30.365427 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:30.365307 5258 generic.go:356] "Generic (PLEG): container finished" podID="415eeabb-bacf-453b-801c-cf123ac99615" containerID="51e0020a215b43de4affc2b6029dbf4b58e6b73d028bfa9a7faa3ed9a58cce2b" exitCode=0 Nov 04 05:46:30.365932 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:30.365467 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-df357346-kn6w8" event={"ID":"415eeabb-bacf-453b-801c-cf123ac99615","Type":"ContainerDied","Data":"51e0020a215b43de4affc2b6029dbf4b58e6b73d028bfa9a7faa3ed9a58cce2b"} Nov 04 05:46:31.510633 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:31.510606 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-df357346-kn6w8" Nov 04 05:46:31.618694 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:31.618655 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/415eeabb-bacf-453b-801c-cf123ac99615-kube-api-access-2p964\" (UniqueName: \"kubernetes.io/projected/415eeabb-bacf-453b-801c-cf123ac99615-kube-api-access-2p964\") pod \"415eeabb-bacf-453b-801c-cf123ac99615\" (UID: \"415eeabb-bacf-453b-801c-cf123ac99615\") " Nov 04 05:46:31.618868 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:31.618808 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/415eeabb-bacf-453b-801c-cf123ac99615-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/415eeabb-bacf-453b-801c-cf123ac99615-maas-api-xks-ca\") pod \"415eeabb-bacf-453b-801c-cf123ac99615\" (UID: \"415eeabb-bacf-453b-801c-cf123ac99615\") " Nov 04 05:46:31.618868 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:31.618844 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/415eeabb-bacf-453b-801c-cf123ac99615-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/415eeabb-bacf-453b-801c-cf123ac99615-maas-api-openshift-ca\") pod \"415eeabb-bacf-453b-801c-cf123ac99615\" (UID: \"415eeabb-bacf-453b-801c-cf123ac99615\") " Nov 04 05:46:31.619276 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:31.619253 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/415eeabb-bacf-453b-801c-cf123ac99615-maas-api-openshift-ca" pod "415eeabb-bacf-453b-801c-cf123ac99615" (UID: "415eeabb-bacf-453b-801c-cf123ac99615"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:31.620836 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:31.620815 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/415eeabb-bacf-453b-801c-cf123ac99615-maas-api-xks-ca" pod "415eeabb-bacf-453b-801c-cf123ac99615" (UID: "415eeabb-bacf-453b-801c-cf123ac99615"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:46:31.620889 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:31.620845 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/415eeabb-bacf-453b-801c-cf123ac99615-kube-api-access-2p964" pod "415eeabb-bacf-453b-801c-cf123ac99615" (UID: "415eeabb-bacf-453b-801c-cf123ac99615"). InnerVolumeSpecName "kube-api-access-2p964". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:31.720539 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:31.720502 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/415eeabb-bacf-453b-801c-cf123ac99615-maas-api-xks-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:31.720539 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:31.720533 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/415eeabb-bacf-453b-801c-cf123ac99615-maas-api-openshift-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:31.720539 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:31.720545 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2p964\" (UniqueName: \"kubernetes.io/projected/415eeabb-bacf-453b-801c-cf123ac99615-kube-api-access-2p964\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:32.377522 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:32.377489 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-df357346-kn6w8" Nov 04 05:46:32.377666 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:32.377522 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-df357346-kn6w8" event={"ID":"415eeabb-bacf-453b-801c-cf123ac99615","Type":"ContainerDied","Data":"dfc979588a86a359fdf5edd7cb566b0d36dced51c6435fc1de5b88cca161a59f"} Nov 04 05:46:32.377666 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:32.377568 5258 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dfc979588a86a359fdf5edd7cb566b0d36dced51c6435fc1de5b88cca161a59f" Nov 04 05:46:32.517548 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:32.517510 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79"] Nov 04 05:46:34.187101 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.187067 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8253ce6b-f78vw"] Nov 04 05:46:34.187614 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.187599 5258 cpu_manager.go:414] "containerMap: removing container" podUID="6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" containerName="istio-proxy" Nov 04 05:46:34.187673 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.187616 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" containerName="istio-proxy" Nov 04 05:46:34.187673 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.187649 5258 cpu_manager.go:414] "containerMap: removing container" podUID="415eeabb-bacf-453b-801c-cf123ac99615" containerName="revoke-keys" Nov 04 05:46:34.187673 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.187658 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="415eeabb-bacf-453b-801c-cf123ac99615" containerName="revoke-keys" Nov 04 05:46:34.187762 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.187750 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="415eeabb-bacf-453b-801c-cf123ac99615" containerName="revoke-keys" Nov 04 05:46:34.187795 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.187765 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a3d0b5a-4e73-4939-97f5-f662b5bc6b15" containerName="istio-proxy" Nov 04 05:46:34.191245 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.191229 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8253ce6b-f78vw" Nov 04 05:46:34.252085 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.252051 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 05:46:34.252085 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.252059 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Nov 04 05:46:34.254843 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.254819 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8253ce6b-f78vw"] Nov 04 05:46:34.308472 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.308440 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 05:46:34.308639 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.308530 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dnddh\"" Nov 04 05:46:34.348227 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.348191 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnj9w\" (UniqueName: \"kubernetes.io/projected/d16c63d4-9853-4510-978f-56d07cd7f192-kube-api-access-bnj9w\") pod \"maas-api-revoke-sub-8253ce6b-f78vw\" (UID: \"d16c63d4-9853-4510-978f-56d07cd7f192\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8253ce6b-f78vw" Nov 04 05:46:34.348379 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.348320 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d16c63d4-9853-4510-978f-56d07cd7f192-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8253ce6b-f78vw\" (UID: \"d16c63d4-9853-4510-978f-56d07cd7f192\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8253ce6b-f78vw" Nov 04 05:46:34.348421 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.348401 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d16c63d4-9853-4510-978f-56d07cd7f192-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8253ce6b-f78vw\" (UID: \"d16c63d4-9853-4510-978f-56d07cd7f192\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8253ce6b-f78vw" Nov 04 05:46:34.450011 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.449927 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnj9w\" (UniqueName: \"kubernetes.io/projected/d16c63d4-9853-4510-978f-56d07cd7f192-kube-api-access-bnj9w\") pod \"maas-api-revoke-sub-8253ce6b-f78vw\" (UID: \"d16c63d4-9853-4510-978f-56d07cd7f192\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8253ce6b-f78vw" Nov 04 05:46:34.450205 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.450060 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d16c63d4-9853-4510-978f-56d07cd7f192-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8253ce6b-f78vw\" (UID: \"d16c63d4-9853-4510-978f-56d07cd7f192\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8253ce6b-f78vw" Nov 04 05:46:34.450205 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.450160 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d16c63d4-9853-4510-978f-56d07cd7f192-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8253ce6b-f78vw\" (UID: \"d16c63d4-9853-4510-978f-56d07cd7f192\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8253ce6b-f78vw" Nov 04 05:46:34.450794 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.450765 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d16c63d4-9853-4510-978f-56d07cd7f192-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-8253ce6b-f78vw\" (UID: \"d16c63d4-9853-4510-978f-56d07cd7f192\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8253ce6b-f78vw" Nov 04 05:46:34.452347 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.452316 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d16c63d4-9853-4510-978f-56d07cd7f192-maas-api-xks-ca\") pod \"maas-api-revoke-sub-8253ce6b-f78vw\" (UID: \"d16c63d4-9853-4510-978f-56d07cd7f192\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8253ce6b-f78vw" Nov 04 05:46:34.517112 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.517082 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnj9w\" (UniqueName: \"kubernetes.io/projected/d16c63d4-9853-4510-978f-56d07cd7f192-kube-api-access-bnj9w\") pod \"maas-api-revoke-sub-8253ce6b-f78vw\" (UID: \"d16c63d4-9853-4510-978f-56d07cd7f192\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-8253ce6b-f78vw" Nov 04 05:46:34.804518 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:34.804439 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8253ce6b-f78vw" Nov 04 05:46:35.192824 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:35.192799 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-8253ce6b-f78vw"] Nov 04 05:46:35.195379 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:46:35.195347 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd16c63d4_9853_4510_978f_56d07cd7f192.slice/crio-d4d4ce3986762119eca85e8cb52ec9978b019c7c11414698b1ddb6ec023081ab WatchSource:0}: Error finding container d4d4ce3986762119eca85e8cb52ec9978b019c7c11414698b1ddb6ec023081ab: Status 404 returned error can't find the container with id d4d4ce3986762119eca85e8cb52ec9978b019c7c11414698b1ddb6ec023081ab Nov 04 05:46:35.394447 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:35.394416 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8253ce6b-f78vw" event={"ID":"d16c63d4-9853-4510-978f-56d07cd7f192","Type":"ContainerStarted","Data":"d4d4ce3986762119eca85e8cb52ec9978b019c7c11414698b1ddb6ec023081ab"} Nov 04 05:46:36.401089 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:36.400988 5258 generic.go:356] "Generic (PLEG): container finished" podID="d16c63d4-9853-4510-978f-56d07cd7f192" containerID="7f475af2f75d488891ab984b48b337d58c432600e0fb2e3bfea69d273fbebecf" exitCode=0 Nov 04 05:46:36.401089 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:36.401061 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8253ce6b-f78vw" event={"ID":"d16c63d4-9853-4510-978f-56d07cd7f192","Type":"ContainerDied","Data":"7f475af2f75d488891ab984b48b337d58c432600e0fb2e3bfea69d273fbebecf"} Nov 04 05:46:37.542879 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:37.542853 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8253ce6b-f78vw" Nov 04 05:46:37.582863 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:37.582830 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d16c63d4-9853-4510-978f-56d07cd7f192-kube-api-access-bnj9w\" (UniqueName: \"kubernetes.io/projected/d16c63d4-9853-4510-978f-56d07cd7f192-kube-api-access-bnj9w\") pod \"d16c63d4-9853-4510-978f-56d07cd7f192\" (UID: \"d16c63d4-9853-4510-978f-56d07cd7f192\") " Nov 04 05:46:37.583018 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:37.582927 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d16c63d4-9853-4510-978f-56d07cd7f192-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d16c63d4-9853-4510-978f-56d07cd7f192-maas-api-xks-ca\") pod \"d16c63d4-9853-4510-978f-56d07cd7f192\" (UID: \"d16c63d4-9853-4510-978f-56d07cd7f192\") " Nov 04 05:46:37.583018 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:37.582998 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d16c63d4-9853-4510-978f-56d07cd7f192-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d16c63d4-9853-4510-978f-56d07cd7f192-maas-api-openshift-ca\") pod \"d16c63d4-9853-4510-978f-56d07cd7f192\" (UID: \"d16c63d4-9853-4510-978f-56d07cd7f192\") " Nov 04 05:46:37.583400 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:37.583375 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d16c63d4-9853-4510-978f-56d07cd7f192-maas-api-openshift-ca" pod "d16c63d4-9853-4510-978f-56d07cd7f192" (UID: "d16c63d4-9853-4510-978f-56d07cd7f192"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:37.584950 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:37.584926 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d16c63d4-9853-4510-978f-56d07cd7f192-maas-api-xks-ca" pod "d16c63d4-9853-4510-978f-56d07cd7f192" (UID: "d16c63d4-9853-4510-978f-56d07cd7f192"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:46:37.585053 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:37.584998 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d16c63d4-9853-4510-978f-56d07cd7f192-kube-api-access-bnj9w" pod "d16c63d4-9853-4510-978f-56d07cd7f192" (UID: "d16c63d4-9853-4510-978f-56d07cd7f192"). InnerVolumeSpecName "kube-api-access-bnj9w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:37.684517 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:37.684417 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d16c63d4-9853-4510-978f-56d07cd7f192-maas-api-openshift-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:37.684517 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:37.684447 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnj9w\" (UniqueName: \"kubernetes.io/projected/d16c63d4-9853-4510-978f-56d07cd7f192-kube-api-access-bnj9w\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:37.684517 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:37.684457 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d16c63d4-9853-4510-978f-56d07cd7f192-maas-api-xks-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:38.412977 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:38.412944 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8253ce6b-f78vw" Nov 04 05:46:38.413186 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:38.412973 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-8253ce6b-f78vw" event={"ID":"d16c63d4-9853-4510-978f-56d07cd7f192","Type":"ContainerDied","Data":"d4d4ce3986762119eca85e8cb52ec9978b019c7c11414698b1ddb6ec023081ab"} Nov 04 05:46:38.413186 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:38.413022 5258 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4d4ce3986762119eca85e8cb52ec9978b019c7c11414698b1ddb6ec023081ab" Nov 04 05:46:38.466193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:38.466157 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz"] Nov 04 05:46:38.523014 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:38.522984 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms"] Nov 04 05:46:42.029584 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.029545 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp"] Nov 04 05:46:42.030110 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.029978 5258 cpu_manager.go:414] "containerMap: removing container" podUID="d16c63d4-9853-4510-978f-56d07cd7f192" containerName="revoke-keys" Nov 04 05:46:42.030110 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.029989 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d16c63d4-9853-4510-978f-56d07cd7f192" containerName="revoke-keys" Nov 04 05:46:42.030110 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.030099 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="d16c63d4-9853-4510-978f-56d07cd7f192" containerName="revoke-keys" Nov 04 05:46:42.033466 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.033440 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.126232 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.126191 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.126232 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.126235 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c20521f-5f13-49b4-954d-08b2bc8554d5-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.126469 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.126387 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjwd6\" (UniqueName: \"kubernetes.io/projected/0c20521f-5f13-49b4-954d-08b2bc8554d5-kube-api-access-jjwd6\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.126537 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.126485 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.126537 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.126516 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.126626 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.126549 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.126674 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.126653 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.126706 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.126694 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.126766 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.126745 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.152070 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.152022 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp"] Nov 04 05:46:42.227957 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.227918 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.227957 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.227964 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.228191 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.228081 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c20521f-5f13-49b4-954d-08b2bc8554d5-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.228191 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.228141 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjwd6\" (UniqueName: \"kubernetes.io/projected/0c20521f-5f13-49b4-954d-08b2bc8554d5-kube-api-access-jjwd6\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.228191 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.228183 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.228334 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.228201 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.228334 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.228235 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.228334 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.228284 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.228483 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.228334 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.228483 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.228452 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.228745 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.228715 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.228745 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.228734 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.228920 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.228792 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.228920 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.228805 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c20521f-5f13-49b4-954d-08b2bc8554d5-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.230250 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.230232 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.230650 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.230635 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.296290 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.296214 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.296424 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.296348 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjwd6\" (UniqueName: \"kubernetes.io/projected/0c20521f-5f13-49b4-954d-08b2bc8554d5-kube-api-access-jjwd6\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.345835 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.345809 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:42.790704 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.790669 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp"] Nov 04 05:46:42.794410 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:46:42.794379 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c20521f_5f13_49b4_954d_08b2bc8554d5.slice/crio-5c205838a9a39290f5ec2677a1b670b00d5679616c2fa1dfc1c89d3232397917 WatchSource:0}: Error finding container 5c205838a9a39290f5ec2677a1b670b00d5679616c2fa1dfc1c89d3232397917: Status 404 returned error can't find the container with id 5c205838a9a39290f5ec2677a1b670b00d5679616c2fa1dfc1c89d3232397917 Nov 04 05:46:42.796639 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.796608 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:46:42.796716 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.796669 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:46:42.796716 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:42.796699 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:46:43.463578 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:43.463537 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" event={"ID":"0c20521f-5f13-49b4-954d-08b2bc8554d5","Type":"ContainerStarted","Data":"c30894172b64d0398b1c6f4f465d41e732395708b153386ec578a20264aa433d"} Nov 04 05:46:43.464072 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:43.463591 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" event={"ID":"0c20521f-5f13-49b4-954d-08b2bc8554d5","Type":"ContainerStarted","Data":"5c205838a9a39290f5ec2677a1b670b00d5679616c2fa1dfc1c89d3232397917"} Nov 04 05:46:44.354416 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:44.354382 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:44.356631 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:44.356602 5258 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body= Nov 04 05:46:44.356851 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:44.356823 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" podUID="0c20521f-5f13-49b4-954d-08b2bc8554d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" Nov 04 05:46:45.358061 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:45.353478 5258 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body= Nov 04 05:46:45.358061 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:45.353547 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" podUID="0c20521f-5f13-49b4-954d-08b2bc8554d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" Nov 04 05:46:46.358173 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:46.358135 5258 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body= Nov 04 05:46:46.358583 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:46.358193 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" podUID="0c20521f-5f13-49b4-954d-08b2bc8554d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" Nov 04 05:46:47.349937 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:47.349463 5258 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body= Nov 04 05:46:47.349937 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:47.349542 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" podUID="0c20521f-5f13-49b4-954d-08b2bc8554d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" Nov 04 05:46:47.546064 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:47.546000 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" podUID="964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd" containerName="main" containerID="cri-o://ee786e026d7432e45c73cd3c38654f93c06d644b4855e062775f67bd3203990a" gracePeriod=45 Nov 04 05:46:47.829050 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:47.829008 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:46:47.910813 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:47.910780 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-dshm\" (UniqueName: \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-dshm\") pod \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " Nov 04 05:46:47.910992 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:47.910835 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-model-cache\") pod \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " Nov 04 05:46:47.910992 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:47.910980 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-kube-api-access-wklph\" (UniqueName: \"kubernetes.io/projected/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-kube-api-access-wklph\") pod \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " Nov 04 05:46:47.911126 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:47.911079 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-home\" (UniqueName: \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-home\") pod \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " Nov 04 05:46:47.911249 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:47.911123 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-tls-certs\" (UniqueName: \"kubernetes.io/secret/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-tls-certs\") pod \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " Nov 04 05:46:47.911249 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:47.911195 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-tmp-dir\") pod \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\" (UID: \"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd\") " Nov 04 05:46:47.911356 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:47.911186 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-model-cache" pod "964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd" (UID: "964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:47.911356 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:47.911304 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-home" pod "964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd" (UID: "964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:47.911455 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:47.911436 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-tmp-dir" pod "964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd" (UID: "964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:47.911594 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:47.911574 5258 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-home\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:47.911665 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:47.911610 5258 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-tmp-dir\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:47.911665 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:47.911620 5258 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-model-cache\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:47.913244 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:47.913221 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-dshm" pod "964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd" (UID: "964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:47.913331 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:47.913260 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-tls-certs" pod "964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd" (UID: "964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:46:47.913331 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:47.913274 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-kube-api-access-wklph" pod "964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd" (UID: "964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd"). InnerVolumeSpecName "kube-api-access-wklph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:47.964418 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:47.964302 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" podStartSLOduration=6.964282674 podStartE2EDuration="6.964282674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:46:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:46:43.66150854 +0000 UTC m=+1137.540901024" watchObservedRunningTime="2026-11-04 05:46:47.964282674 +0000 UTC m=+1141.843675148" Nov 04 05:46:48.013090 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:48.013058 5258 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-tls-certs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:48.013090 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:48.013085 5258 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-dshm\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:48.013090 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:48.013097 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wklph\" (UniqueName: \"kubernetes.io/projected/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd-kube-api-access-wklph\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:48.348933 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:48.348459 5258 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body= Nov 04 05:46:48.348933 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:48.348525 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" podUID="0c20521f-5f13-49b4-954d-08b2bc8554d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" Nov 04 05:46:48.498730 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:48.498693 5258 generic.go:356] "Generic (PLEG): container finished" podID="964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd" containerID="ee786e026d7432e45c73cd3c38654f93c06d644b4855e062775f67bd3203990a" exitCode=0 Nov 04 05:46:48.498888 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:48.498742 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" Nov 04 05:46:48.498888 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:48.498772 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" event={"ID":"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd","Type":"ContainerDied","Data":"ee786e026d7432e45c73cd3c38654f93c06d644b4855e062775f67bd3203990a"} Nov 04 05:46:48.498888 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:48.498820 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79" event={"ID":"964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd","Type":"ContainerDied","Data":"6e7919c5782d5360251fac950dab3806e03f3017ddcf49a1c16d81b8b352d999"} Nov 04 05:46:48.498888 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:48.498836 5258 scope.go:122] "RemoveContainer" containerID="ee786e026d7432e45c73cd3c38654f93c06d644b4855e062775f67bd3203990a" Nov 04 05:46:48.508952 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:48.508935 5258 scope.go:122] "RemoveContainer" containerID="ee786e026d7432e45c73cd3c38654f93c06d644b4855e062775f67bd3203990a" Nov 04 05:46:48.509245 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:46:48.509223 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee786e026d7432e45c73cd3c38654f93c06d644b4855e062775f67bd3203990a\": container with ID starting with ee786e026d7432e45c73cd3c38654f93c06d644b4855e062775f67bd3203990a not found: ID does not exist" containerID="ee786e026d7432e45c73cd3c38654f93c06d644b4855e062775f67bd3203990a" Nov 04 05:46:48.509348 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:48.509251 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee786e026d7432e45c73cd3c38654f93c06d644b4855e062775f67bd3203990a"} err="failed to get container status \"ee786e026d7432e45c73cd3c38654f93c06d644b4855e062775f67bd3203990a\": rpc error: code = NotFound desc = could not find container \"ee786e026d7432e45c73cd3c38654f93c06d644b4855e062775f67bd3203990a\": container with ID starting with ee786e026d7432e45c73cd3c38654f93c06d644b4855e062775f67bd3203990a not found: ID does not exist" Nov 04 05:46:48.685093 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:48.685054 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79"] Nov 04 05:46:48.741142 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:48.741110 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-961485/distinct-2-w2-961485-kserve-588d499c6f-jdj79"] Nov 04 05:46:48.843085 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:48.843053 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:46:49.150973 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.150941 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4decd865-zfldm"] Nov 04 05:46:49.151489 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.151471 5258 cpu_manager.go:414] "containerMap: removing container" podUID="964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd" containerName="main" Nov 04 05:46:49.151588 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.151491 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd" containerName="main" Nov 04 05:46:49.151649 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.151588 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd" containerName="main" Nov 04 05:46:49.156203 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.156183 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4decd865-zfldm" Nov 04 05:46:49.225494 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.223689 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4decd865-zfldm"] Nov 04 05:46:49.225494 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.223830 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dnddh\"" Nov 04 05:46:49.225494 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.223894 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Nov 04 05:46:49.225494 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.223958 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 05:46:49.225494 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.224693 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e0bb9631-1874-471a-8c19-6dc5166d75ba-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4decd865-zfldm\" (UID: \"e0bb9631-1874-471a-8c19-6dc5166d75ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4decd865-zfldm" Nov 04 05:46:49.225494 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.224749 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sftht\" (UniqueName: \"kubernetes.io/projected/e0bb9631-1874-471a-8c19-6dc5166d75ba-kube-api-access-sftht\") pod \"maas-api-revoke-sub-4decd865-zfldm\" (UID: \"e0bb9631-1874-471a-8c19-6dc5166d75ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4decd865-zfldm" Nov 04 05:46:49.225494 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.224821 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e0bb9631-1874-471a-8c19-6dc5166d75ba-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4decd865-zfldm\" (UID: \"e0bb9631-1874-471a-8c19-6dc5166d75ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4decd865-zfldm" Nov 04 05:46:49.226327 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.226299 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 05:46:49.326243 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.326204 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e0bb9631-1874-471a-8c19-6dc5166d75ba-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4decd865-zfldm\" (UID: \"e0bb9631-1874-471a-8c19-6dc5166d75ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4decd865-zfldm" Nov 04 05:46:49.326453 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.326268 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e0bb9631-1874-471a-8c19-6dc5166d75ba-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4decd865-zfldm\" (UID: \"e0bb9631-1874-471a-8c19-6dc5166d75ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4decd865-zfldm" Nov 04 05:46:49.326453 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.326309 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sftht\" (UniqueName: \"kubernetes.io/projected/e0bb9631-1874-471a-8c19-6dc5166d75ba-kube-api-access-sftht\") pod \"maas-api-revoke-sub-4decd865-zfldm\" (UID: \"e0bb9631-1874-471a-8c19-6dc5166d75ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4decd865-zfldm" Nov 04 05:46:49.326975 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.326950 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e0bb9631-1874-471a-8c19-6dc5166d75ba-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4decd865-zfldm\" (UID: \"e0bb9631-1874-471a-8c19-6dc5166d75ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4decd865-zfldm" Nov 04 05:46:49.328871 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.328835 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e0bb9631-1874-471a-8c19-6dc5166d75ba-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4decd865-zfldm\" (UID: \"e0bb9631-1874-471a-8c19-6dc5166d75ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4decd865-zfldm" Nov 04 05:46:49.347001 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.346970 5258 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body= Nov 04 05:46:49.347147 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.347046 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" podUID="0c20521f-5f13-49b4-954d-08b2bc8554d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" Nov 04 05:46:49.395019 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.394981 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sftht\" (UniqueName: \"kubernetes.io/projected/e0bb9631-1874-471a-8c19-6dc5166d75ba-kube-api-access-sftht\") pod \"maas-api-revoke-sub-4decd865-zfldm\" (UID: \"e0bb9631-1874-471a-8c19-6dc5166d75ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4decd865-zfldm" Nov 04 05:46:49.469471 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.469380 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4decd865-zfldm" Nov 04 05:46:49.886221 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:49.886169 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4decd865-zfldm"] Nov 04 05:46:49.906413 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:46:49.906382 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0bb9631_1874_471a_8c19_6dc5166d75ba.slice/crio-28963b692c40d7d3a33deb35fc45d6cc61302d0137a2037d3ed87bc8656e6649 WatchSource:0}: Error finding container 28963b692c40d7d3a33deb35fc45d6cc61302d0137a2037d3ed87bc8656e6649: Status 404 returned error can't find the container with id 28963b692c40d7d3a33deb35fc45d6cc61302d0137a2037d3ed87bc8656e6649 Nov 04 05:46:50.351061 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:50.334375 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd" path="/var/lib/kubelet/pods/964baa93-a9cb-4d82-9ee8-4ca1d7cfc8fd/volumes" Nov 04 05:46:50.530913 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:50.530836 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4decd865-zfldm" event={"ID":"e0bb9631-1874-471a-8c19-6dc5166d75ba","Type":"ContainerStarted","Data":"28963b692c40d7d3a33deb35fc45d6cc61302d0137a2037d3ed87bc8656e6649"} Nov 04 05:46:50.915008 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:50.914632 5258 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 05:46:50.915008 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:50.914700 5258 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" podUID="33978c82-e069-4b5c-bbe9-049f35fb31af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 05:46:51.353350 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:51.353300 5258 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 05:46:51.353562 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:51.353379 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" podUID="0c20521f-5f13-49b4-954d-08b2bc8554d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 05:46:51.363402 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:51.363362 5258 patch_prober.go:29] interesting pod/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 05:46:51.363584 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:51.363429 5258 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" podUID="8aa179d9-60b7-4dec-a891-a379cb56e798" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 05:46:51.541284 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:51.541185 5258 generic.go:356] "Generic (PLEG): container finished" podID="e0bb9631-1874-471a-8c19-6dc5166d75ba" containerID="ba6e8a73e12b15ace91cd024350e98a920dbac6ab85916bd284ac233c68622b8" exitCode=0 Nov 04 05:46:51.541284 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:51.541248 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4decd865-zfldm" event={"ID":"e0bb9631-1874-471a-8c19-6dc5166d75ba","Type":"ContainerDied","Data":"ba6e8a73e12b15ace91cd024350e98a920dbac6ab85916bd284ac233c68622b8"} Nov 04 05:46:51.934120 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:51.934086 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k"] Nov 04 05:46:52.049794 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:52.049746 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf"] Nov 04 05:46:52.148094 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:52.148059 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:52.148542 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:52.148505 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:52.149404 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:52.149384 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:46:52.687844 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:52.687821 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4decd865-zfldm" Nov 04 05:46:52.781187 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:46:52.781154 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:46:52.812674 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:52.812588 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e0bb9631-1874-471a-8c19-6dc5166d75ba-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e0bb9631-1874-471a-8c19-6dc5166d75ba-maas-api-openshift-ca\") pod \"e0bb9631-1874-471a-8c19-6dc5166d75ba\" (UID: \"e0bb9631-1874-471a-8c19-6dc5166d75ba\") " Nov 04 05:46:52.812836 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:52.812703 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e0bb9631-1874-471a-8c19-6dc5166d75ba-kube-api-access-sftht\" (UniqueName: \"kubernetes.io/projected/e0bb9631-1874-471a-8c19-6dc5166d75ba-kube-api-access-sftht\") pod \"e0bb9631-1874-471a-8c19-6dc5166d75ba\" (UID: \"e0bb9631-1874-471a-8c19-6dc5166d75ba\") " Nov 04 05:46:52.812836 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:52.812731 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e0bb9631-1874-471a-8c19-6dc5166d75ba-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e0bb9631-1874-471a-8c19-6dc5166d75ba-maas-api-xks-ca\") pod \"e0bb9631-1874-471a-8c19-6dc5166d75ba\" (UID: \"e0bb9631-1874-471a-8c19-6dc5166d75ba\") " Nov 04 05:46:52.813065 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:52.813013 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0bb9631-1874-471a-8c19-6dc5166d75ba-maas-api-openshift-ca" pod "e0bb9631-1874-471a-8c19-6dc5166d75ba" (UID: "e0bb9631-1874-471a-8c19-6dc5166d75ba"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:52.815059 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:52.814846 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0bb9631-1874-471a-8c19-6dc5166d75ba-kube-api-access-sftht" pod "e0bb9631-1874-471a-8c19-6dc5166d75ba" (UID: "e0bb9631-1874-471a-8c19-6dc5166d75ba"). InnerVolumeSpecName "kube-api-access-sftht". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:52.815059 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:52.814855 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0bb9631-1874-471a-8c19-6dc5166d75ba-maas-api-xks-ca" pod "e0bb9631-1874-471a-8c19-6dc5166d75ba" (UID: "e0bb9631-1874-471a-8c19-6dc5166d75ba"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:46:52.914313 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:52.914279 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e0bb9631-1874-471a-8c19-6dc5166d75ba-maas-api-openshift-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:52.914313 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:52.914306 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sftht\" (UniqueName: \"kubernetes.io/projected/e0bb9631-1874-471a-8c19-6dc5166d75ba-kube-api-access-sftht\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:52.914313 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:52.914317 5258 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e0bb9631-1874-471a-8c19-6dc5166d75ba-maas-api-xks-ca\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:53.486380 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:53.486333 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" podUID="9e3dfd3b-ab7f-4872-b247-9cfeb0456112" containerName="main" containerID="cri-o://c96230dadb42f30d18cd7bacaec2cedec98c0408610d4e468786c539a1ace0c8" gracePeriod=45 Nov 04 05:46:53.542413 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:53.542369 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" podUID="ac5659ba-c807-44de-83f3-89ccd5eeac3b" containerName="main" containerID="cri-o://92cd0ad91e3f3e233fb34ae1c79f2add4d3efb0ed9019ef85abdd186d0796b46" gracePeriod=45 Nov 04 05:46:53.556476 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:53.556456 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4decd865-zfldm" Nov 04 05:46:53.556587 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:53.556537 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4decd865-zfldm" event={"ID":"e0bb9631-1874-471a-8c19-6dc5166d75ba","Type":"ContainerDied","Data":"28963b692c40d7d3a33deb35fc45d6cc61302d0137a2037d3ed87bc8656e6649"} Nov 04 05:46:53.556587 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:53.556578 5258 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28963b692c40d7d3a33deb35fc45d6cc61302d0137a2037d3ed87bc8656e6649" Nov 04 05:46:53.832673 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:53.832648 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:46:53.854331 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:53.854308 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:46:53.925493 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:53.925455 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-tmp-dir\") pod \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " Nov 04 05:46:53.925647 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:53.925510 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-tls-certs\" (UniqueName: \"kubernetes.io/secret/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-tls-certs\") pod \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " Nov 04 05:46:53.925647 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:53.925579 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-home\" (UniqueName: \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-home\") pod \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " Nov 04 05:46:53.925774 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:53.925666 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-dshm\") pod \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " Nov 04 05:46:53.925774 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:53.925705 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-kube-api-access-bfqjh\" (UniqueName: \"kubernetes.io/projected/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-kube-api-access-bfqjh\") pod \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " Nov 04 05:46:53.925774 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:53.925763 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-model-cache\") pod \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\" (UID: \"9e3dfd3b-ab7f-4872-b247-9cfeb0456112\") " Nov 04 05:46:53.925940 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:53.925787 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-tmp-dir" pod "9e3dfd3b-ab7f-4872-b247-9cfeb0456112" (UID: "9e3dfd3b-ab7f-4872-b247-9cfeb0456112"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:53.925940 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:53.925893 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-home" pod "9e3dfd3b-ab7f-4872-b247-9cfeb0456112" (UID: "9e3dfd3b-ab7f-4872-b247-9cfeb0456112"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:53.926105 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:53.926081 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-model-cache" pod "9e3dfd3b-ab7f-4872-b247-9cfeb0456112" (UID: "9e3dfd3b-ab7f-4872-b247-9cfeb0456112"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:53.926218 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:53.926193 5258 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-tmp-dir\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:53.926218 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:53.926218 5258 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-home\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:53.926379 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:53.926230 5258 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-model-cache\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:53.928159 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:53.928131 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-kube-api-access-bfqjh" pod "9e3dfd3b-ab7f-4872-b247-9cfeb0456112" (UID: "9e3dfd3b-ab7f-4872-b247-9cfeb0456112"). InnerVolumeSpecName "kube-api-access-bfqjh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:53.928262 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:53.928139 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-dshm" pod "9e3dfd3b-ab7f-4872-b247-9cfeb0456112" (UID: "9e3dfd3b-ab7f-4872-b247-9cfeb0456112"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:53.928262 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:53.928213 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-tls-certs" pod "9e3dfd3b-ab7f-4872-b247-9cfeb0456112" (UID: "9e3dfd3b-ab7f-4872-b247-9cfeb0456112"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:46:54.026915 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.026836 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ac5659ba-c807-44de-83f3-89ccd5eeac3b-tls-certs\" (UniqueName: \"kubernetes.io/secret/ac5659ba-c807-44de-83f3-89ccd5eeac3b-tls-certs\") pod \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " Nov 04 05:46:54.027086 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.026958 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-dshm\") pod \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " Nov 04 05:46:54.027086 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.027096 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-model-cache\") pod \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " Nov 04 05:46:54.027273 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.027136 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-tmp-dir\") pod \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " Nov 04 05:46:54.027273 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.027182 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac5659ba-c807-44de-83f3-89ccd5eeac3b-kube-api-access-hl42r\" (UniqueName: \"kubernetes.io/projected/ac5659ba-c807-44de-83f3-89ccd5eeac3b-kube-api-access-hl42r\") pod \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " Nov 04 05:46:54.027273 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.027242 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-home\" (UniqueName: \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-home\") pod \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\" (UID: \"ac5659ba-c807-44de-83f3-89ccd5eeac3b\") " Nov 04 05:46:54.027453 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.027382 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-model-cache" pod "ac5659ba-c807-44de-83f3-89ccd5eeac3b" (UID: "ac5659ba-c807-44de-83f3-89ccd5eeac3b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:54.027453 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.027401 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-tmp-dir" pod "ac5659ba-c807-44de-83f3-89ccd5eeac3b" (UID: "ac5659ba-c807-44de-83f3-89ccd5eeac3b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:54.027626 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.027605 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-home" pod "ac5659ba-c807-44de-83f3-89ccd5eeac3b" (UID: "ac5659ba-c807-44de-83f3-89ccd5eeac3b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:54.027692 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.027662 5258 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-model-cache\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:54.027692 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.027684 5258 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-tmp-dir\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:54.027794 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.027699 5258 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-dshm\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:54.027794 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.027716 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfqjh\" (UniqueName: \"kubernetes.io/projected/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-kube-api-access-bfqjh\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:54.027794 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.027731 5258 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e3dfd3b-ab7f-4872-b247-9cfeb0456112-tls-certs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:54.029211 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.029187 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-dshm" pod "ac5659ba-c807-44de-83f3-89ccd5eeac3b" (UID: "ac5659ba-c807-44de-83f3-89ccd5eeac3b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:46:54.029752 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.029731 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac5659ba-c807-44de-83f3-89ccd5eeac3b-tls-certs" pod "ac5659ba-c807-44de-83f3-89ccd5eeac3b" (UID: "ac5659ba-c807-44de-83f3-89ccd5eeac3b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 05:46:54.029831 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.029731 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac5659ba-c807-44de-83f3-89ccd5eeac3b-kube-api-access-hl42r" pod "ac5659ba-c807-44de-83f3-89ccd5eeac3b" (UID: "ac5659ba-c807-44de-83f3-89ccd5eeac3b"). InnerVolumeSpecName "kube-api-access-hl42r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:54.129196 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.129163 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hl42r\" (UniqueName: \"kubernetes.io/projected/ac5659ba-c807-44de-83f3-89ccd5eeac3b-kube-api-access-hl42r\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:54.129196 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.129193 5258 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-home\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:54.129385 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.129207 5258 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac5659ba-c807-44de-83f3-89ccd5eeac3b-tls-certs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:54.129385 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.129216 5258 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac5659ba-c807-44de-83f3-89ccd5eeac3b-dshm\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:54.563737 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.563660 5258 generic.go:356] "Generic (PLEG): container finished" podID="ac5659ba-c807-44de-83f3-89ccd5eeac3b" containerID="92cd0ad91e3f3e233fb34ae1c79f2add4d3efb0ed9019ef85abdd186d0796b46" exitCode=0 Nov 04 05:46:54.563737 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.563702 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" Nov 04 05:46:54.564233 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.563737 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" event={"ID":"ac5659ba-c807-44de-83f3-89ccd5eeac3b","Type":"ContainerDied","Data":"92cd0ad91e3f3e233fb34ae1c79f2add4d3efb0ed9019ef85abdd186d0796b46"} Nov 04 05:46:54.564233 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.563781 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms" event={"ID":"ac5659ba-c807-44de-83f3-89ccd5eeac3b","Type":"ContainerDied","Data":"76582ffc158a56e0140731d72219a75a217d739e486f1efb9bb6b516617d577e"} Nov 04 05:46:54.564233 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.563806 5258 scope.go:122] "RemoveContainer" containerID="92cd0ad91e3f3e233fb34ae1c79f2add4d3efb0ed9019ef85abdd186d0796b46" Nov 04 05:46:54.565390 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.565367 5258 generic.go:356] "Generic (PLEG): container finished" podID="9e3dfd3b-ab7f-4872-b247-9cfeb0456112" containerID="c96230dadb42f30d18cd7bacaec2cedec98c0408610d4e468786c539a1ace0c8" exitCode=0 Nov 04 05:46:54.565510 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.565397 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" Nov 04 05:46:54.565510 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.565482 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" event={"ID":"9e3dfd3b-ab7f-4872-b247-9cfeb0456112","Type":"ContainerDied","Data":"c96230dadb42f30d18cd7bacaec2cedec98c0408610d4e468786c539a1ace0c8"} Nov 04 05:46:54.565626 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.565511 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz" event={"ID":"9e3dfd3b-ab7f-4872-b247-9cfeb0456112","Type":"ContainerDied","Data":"bc504024878d656aa59fe9573930af32728d3d712fc0574c6c02b94b40bbc3a4"} Nov 04 05:46:54.575629 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.575608 5258 scope.go:122] "RemoveContainer" containerID="92cd0ad91e3f3e233fb34ae1c79f2add4d3efb0ed9019ef85abdd186d0796b46" Nov 04 05:46:54.576026 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:46:54.576002 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92cd0ad91e3f3e233fb34ae1c79f2add4d3efb0ed9019ef85abdd186d0796b46\": container with ID starting with 92cd0ad91e3f3e233fb34ae1c79f2add4d3efb0ed9019ef85abdd186d0796b46 not found: ID does not exist" containerID="92cd0ad91e3f3e233fb34ae1c79f2add4d3efb0ed9019ef85abdd186d0796b46" Nov 04 05:46:54.576141 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.576057 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92cd0ad91e3f3e233fb34ae1c79f2add4d3efb0ed9019ef85abdd186d0796b46"} err="failed to get container status \"92cd0ad91e3f3e233fb34ae1c79f2add4d3efb0ed9019ef85abdd186d0796b46\": rpc error: code = NotFound desc = could not find container \"92cd0ad91e3f3e233fb34ae1c79f2add4d3efb0ed9019ef85abdd186d0796b46\": container with ID starting with 92cd0ad91e3f3e233fb34ae1c79f2add4d3efb0ed9019ef85abdd186d0796b46 not found: ID does not exist" Nov 04 05:46:54.576141 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.576077 5258 scope.go:122] "RemoveContainer" containerID="c96230dadb42f30d18cd7bacaec2cedec98c0408610d4e468786c539a1ace0c8" Nov 04 05:46:54.584364 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.584347 5258 scope.go:122] "RemoveContainer" containerID="c96230dadb42f30d18cd7bacaec2cedec98c0408610d4e468786c539a1ace0c8" Nov 04 05:46:54.584622 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:46:54.584606 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c96230dadb42f30d18cd7bacaec2cedec98c0408610d4e468786c539a1ace0c8\": container with ID starting with c96230dadb42f30d18cd7bacaec2cedec98c0408610d4e468786c539a1ace0c8 not found: ID does not exist" containerID="c96230dadb42f30d18cd7bacaec2cedec98c0408610d4e468786c539a1ace0c8" Nov 04 05:46:54.584673 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.584629 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c96230dadb42f30d18cd7bacaec2cedec98c0408610d4e468786c539a1ace0c8"} err="failed to get container status \"c96230dadb42f30d18cd7bacaec2cedec98c0408610d4e468786c539a1ace0c8\": rpc error: code = NotFound desc = could not find container \"c96230dadb42f30d18cd7bacaec2cedec98c0408610d4e468786c539a1ace0c8\": container with ID starting with c96230dadb42f30d18cd7bacaec2cedec98c0408610d4e468786c539a1ace0c8 not found: ID does not exist" Nov 04 05:46:54.751359 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.751327 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms"] Nov 04 05:46:54.807690 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.807655 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-901a00/simulator-w4-901a00-kserve-658dfd7ddb-5wqms"] Nov 04 05:46:54.876270 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.876235 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz"] Nov 04 05:46:54.882561 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:54.882536 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-901a00/distinct-2-w4-901a00-kserve-549f59c9df-ktcmz"] Nov 04 05:46:56.269944 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:56.269912 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9e3dfd3b-ab7f-4872-b247-9cfeb0456112" path="/var/lib/kubelet/pods/9e3dfd3b-ab7f-4872-b247-9cfeb0456112/volumes" Nov 04 05:46:56.270366 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:56.270354 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ac5659ba-c807-44de-83f3-89ccd5eeac3b" path="/var/lib/kubelet/pods/ac5659ba-c807-44de-83f3-89ccd5eeac3b/volumes" Nov 04 05:46:56.935275 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:56.935227 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k" podUID="572c549a-0b75-4f41-8bf5-a502b67fdf78" containerName="payload-processing" containerID="cri-o://6a695e4f4fa29b369eb910ed9123b7b7327f442f34506ac1ed19214c87633d56" gracePeriod=25 Nov 04 05:46:57.050269 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.050235 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf" podUID="b1f775db-6655-46cf-8283-e25689017416" containerName="payload-pre-processing" containerID="cri-o://ab76d1530bfcf46e1c70ffdd7e3b834e70817bab17efb21527e4d74a94144bb9" gracePeriod=25 Nov 04 05:46:57.193527 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.193469 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k" Nov 04 05:46:57.296094 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.296072 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf" Nov 04 05:46:57.359087 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.359051 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/572c549a-0b75-4f41-8bf5-a502b67fdf78-kube-api-access-n5j96\" (UniqueName: \"kubernetes.io/projected/572c549a-0b75-4f41-8bf5-a502b67fdf78-kube-api-access-n5j96\") pod \"572c549a-0b75-4f41-8bf5-a502b67fdf78\" (UID: \"572c549a-0b75-4f41-8bf5-a502b67fdf78\") " Nov 04 05:46:57.359250 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.359212 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/572c549a-0b75-4f41-8bf5-a502b67fdf78-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/572c549a-0b75-4f41-8bf5-a502b67fdf78-plugins-config-volume\") pod \"572c549a-0b75-4f41-8bf5-a502b67fdf78\" (UID: \"572c549a-0b75-4f41-8bf5-a502b67fdf78\") " Nov 04 05:46:57.359565 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.359539 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/572c549a-0b75-4f41-8bf5-a502b67fdf78-plugins-config-volume" pod "572c549a-0b75-4f41-8bf5-a502b67fdf78" (UID: "572c549a-0b75-4f41-8bf5-a502b67fdf78"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:57.361293 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.361272 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/572c549a-0b75-4f41-8bf5-a502b67fdf78-kube-api-access-n5j96" pod "572c549a-0b75-4f41-8bf5-a502b67fdf78" (UID: "572c549a-0b75-4f41-8bf5-a502b67fdf78"). InnerVolumeSpecName "kube-api-access-n5j96". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:57.460216 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.460134 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1f775db-6655-46cf-8283-e25689017416-kube-api-access-r8bjc\" (UniqueName: \"kubernetes.io/projected/b1f775db-6655-46cf-8283-e25689017416-kube-api-access-r8bjc\") pod \"b1f775db-6655-46cf-8283-e25689017416\" (UID: \"b1f775db-6655-46cf-8283-e25689017416\") " Nov 04 05:46:57.460216 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.460177 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b1f775db-6655-46cf-8283-e25689017416-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1f775db-6655-46cf-8283-e25689017416-plugins-config-volume\") pod \"b1f775db-6655-46cf-8283-e25689017416\" (UID: \"b1f775db-6655-46cf-8283-e25689017416\") " Nov 04 05:46:57.460634 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.460615 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5j96\" (UniqueName: \"kubernetes.io/projected/572c549a-0b75-4f41-8bf5-a502b67fdf78-kube-api-access-n5j96\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:57.460698 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.460648 5258 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/572c549a-0b75-4f41-8bf5-a502b67fdf78-plugins-config-volume\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:57.460698 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.460646 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1f775db-6655-46cf-8283-e25689017416-plugins-config-volume" pod "b1f775db-6655-46cf-8283-e25689017416" (UID: "b1f775db-6655-46cf-8283-e25689017416"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:46:57.462334 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.462311 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1f775db-6655-46cf-8283-e25689017416-kube-api-access-r8bjc" pod "b1f775db-6655-46cf-8283-e25689017416" (UID: "b1f775db-6655-46cf-8283-e25689017416"). InnerVolumeSpecName "kube-api-access-r8bjc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:46:57.561197 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.561165 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8bjc\" (UniqueName: \"kubernetes.io/projected/b1f775db-6655-46cf-8283-e25689017416-kube-api-access-r8bjc\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:57.561197 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.561189 5258 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1f775db-6655-46cf-8283-e25689017416-plugins-config-volume\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:46:57.584265 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.584234 5258 generic.go:356] "Generic (PLEG): container finished" podID="b1f775db-6655-46cf-8283-e25689017416" containerID="ab76d1530bfcf46e1c70ffdd7e3b834e70817bab17efb21527e4d74a94144bb9" exitCode=0 Nov 04 05:46:57.584408 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.584283 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf" Nov 04 05:46:57.584408 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.584320 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf" event={"ID":"b1f775db-6655-46cf-8283-e25689017416","Type":"ContainerDied","Data":"ab76d1530bfcf46e1c70ffdd7e3b834e70817bab17efb21527e4d74a94144bb9"} Nov 04 05:46:57.584408 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.584382 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf" event={"ID":"b1f775db-6655-46cf-8283-e25689017416","Type":"ContainerDied","Data":"2c805d39105686a7cd5e26ef47722b0bbe7da7999cf87805f3b09d6a3f53a662"} Nov 04 05:46:57.584580 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.584410 5258 scope.go:122] "RemoveContainer" containerID="ab76d1530bfcf46e1c70ffdd7e3b834e70817bab17efb21527e4d74a94144bb9" Nov 04 05:46:57.585659 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.585636 5258 generic.go:356] "Generic (PLEG): container finished" podID="572c549a-0b75-4f41-8bf5-a502b67fdf78" containerID="6a695e4f4fa29b369eb910ed9123b7b7327f442f34506ac1ed19214c87633d56" exitCode=0 Nov 04 05:46:57.585770 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.585672 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k" Nov 04 05:46:57.585770 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.585687 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k" event={"ID":"572c549a-0b75-4f41-8bf5-a502b67fdf78","Type":"ContainerDied","Data":"6a695e4f4fa29b369eb910ed9123b7b7327f442f34506ac1ed19214c87633d56"} Nov 04 05:46:57.585770 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.585720 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k" event={"ID":"572c549a-0b75-4f41-8bf5-a502b67fdf78","Type":"ContainerDied","Data":"a17047d76f7b807052116ea0c6123904a2e80fb5e6a33c1bfc76682defec1bae"} Nov 04 05:46:57.594272 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.594257 5258 scope.go:122] "RemoveContainer" containerID="ab76d1530bfcf46e1c70ffdd7e3b834e70817bab17efb21527e4d74a94144bb9" Nov 04 05:46:57.594567 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:46:57.594535 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab76d1530bfcf46e1c70ffdd7e3b834e70817bab17efb21527e4d74a94144bb9\": container with ID starting with ab76d1530bfcf46e1c70ffdd7e3b834e70817bab17efb21527e4d74a94144bb9 not found: ID does not exist" containerID="ab76d1530bfcf46e1c70ffdd7e3b834e70817bab17efb21527e4d74a94144bb9" Nov 04 05:46:57.594627 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.594574 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab76d1530bfcf46e1c70ffdd7e3b834e70817bab17efb21527e4d74a94144bb9"} err="failed to get container status \"ab76d1530bfcf46e1c70ffdd7e3b834e70817bab17efb21527e4d74a94144bb9\": rpc error: code = NotFound desc = could not find container \"ab76d1530bfcf46e1c70ffdd7e3b834e70817bab17efb21527e4d74a94144bb9\": container with ID starting with ab76d1530bfcf46e1c70ffdd7e3b834e70817bab17efb21527e4d74a94144bb9 not found: ID does not exist" Nov 04 05:46:57.594627 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.594599 5258 scope.go:122] "RemoveContainer" containerID="6a695e4f4fa29b369eb910ed9123b7b7327f442f34506ac1ed19214c87633d56" Nov 04 05:46:57.602841 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.602824 5258 scope.go:122] "RemoveContainer" containerID="6a695e4f4fa29b369eb910ed9123b7b7327f442f34506ac1ed19214c87633d56" Nov 04 05:46:57.603113 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:46:57.603090 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a695e4f4fa29b369eb910ed9123b7b7327f442f34506ac1ed19214c87633d56\": container with ID starting with 6a695e4f4fa29b369eb910ed9123b7b7327f442f34506ac1ed19214c87633d56 not found: ID does not exist" containerID="6a695e4f4fa29b369eb910ed9123b7b7327f442f34506ac1ed19214c87633d56" Nov 04 05:46:57.603209 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.603116 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a695e4f4fa29b369eb910ed9123b7b7327f442f34506ac1ed19214c87633d56"} err="failed to get container status \"6a695e4f4fa29b369eb910ed9123b7b7327f442f34506ac1ed19214c87633d56\": rpc error: code = NotFound desc = could not find container \"6a695e4f4fa29b369eb910ed9123b7b7327f442f34506ac1ed19214c87633d56\": container with ID starting with 6a695e4f4fa29b369eb910ed9123b7b7327f442f34506ac1ed19214c87633d56 not found: ID does not exist" Nov 04 05:46:57.770781 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.770687 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf"] Nov 04 05:46:57.772835 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.772806 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-901a00-bc6cf6d67-gx5bf"] Nov 04 05:46:57.894972 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.894931 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k"] Nov 04 05:46:57.899253 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:57.899228 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-901a00-86cc8cfdc6-rbw8k"] Nov 04 05:46:58.269953 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:58.269921 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="572c549a-0b75-4f41-8bf5-a502b67fdf78" path="/var/lib/kubelet/pods/572c549a-0b75-4f41-8bf5-a502b67fdf78/volumes" Nov 04 05:46:58.270325 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:46:58.270311 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b1f775db-6655-46cf-8283-e25689017416" path="/var/lib/kubelet/pods/b1f775db-6655-46cf-8283-e25689017416/volumes" Nov 04 05:47:12.314120 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.314076 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw"] Nov 04 05:47:12.314684 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.314663 5258 cpu_manager.go:414] "containerMap: removing container" podUID="b1f775db-6655-46cf-8283-e25689017416" containerName="payload-pre-processing" Nov 04 05:47:12.314791 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.314686 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1f775db-6655-46cf-8283-e25689017416" containerName="payload-pre-processing" Nov 04 05:47:12.314791 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.314701 5258 cpu_manager.go:414] "containerMap: removing container" podUID="572c549a-0b75-4f41-8bf5-a502b67fdf78" containerName="payload-processing" Nov 04 05:47:12.314791 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.314708 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="572c549a-0b75-4f41-8bf5-a502b67fdf78" containerName="payload-processing" Nov 04 05:47:12.314791 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.314739 5258 cpu_manager.go:414] "containerMap: removing container" podUID="e0bb9631-1874-471a-8c19-6dc5166d75ba" containerName="revoke-keys" Nov 04 05:47:12.314791 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.314747 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e0bb9631-1874-471a-8c19-6dc5166d75ba" containerName="revoke-keys" Nov 04 05:47:12.314791 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.314771 5258 cpu_manager.go:414] "containerMap: removing container" podUID="9e3dfd3b-ab7f-4872-b247-9cfeb0456112" containerName="main" Nov 04 05:47:12.314791 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.314778 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e3dfd3b-ab7f-4872-b247-9cfeb0456112" containerName="main" Nov 04 05:47:12.315170 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.314802 5258 cpu_manager.go:414] "containerMap: removing container" podUID="ac5659ba-c807-44de-83f3-89ccd5eeac3b" containerName="main" Nov 04 05:47:12.315170 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.314808 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac5659ba-c807-44de-83f3-89ccd5eeac3b" containerName="main" Nov 04 05:47:12.315170 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.314906 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac5659ba-c807-44de-83f3-89ccd5eeac3b" containerName="main" Nov 04 05:47:12.315170 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.314922 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1f775db-6655-46cf-8283-e25689017416" containerName="payload-pre-processing" Nov 04 05:47:12.315170 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.314932 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="572c549a-0b75-4f41-8bf5-a502b67fdf78" containerName="payload-processing" Nov 04 05:47:12.315170 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.314944 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="e0bb9631-1874-471a-8c19-6dc5166d75ba" containerName="revoke-keys" Nov 04 05:47:12.315170 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.314956 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e3dfd3b-ab7f-4872-b247-9cfeb0456112" containerName="main" Nov 04 05:47:12.318486 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.318463 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.400735 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.400698 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.400735 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.400739 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.400960 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.400758 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf491066-1777-4392-9c3c-a2348b0e7392-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.400960 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.400876 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.400960 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.400914 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.400960 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.400931 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfgdv\" (UniqueName: \"kubernetes.io/projected/cf491066-1777-4392-9c3c-a2348b0e7392-kube-api-access-bfgdv\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.401113 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.401053 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.401113 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.401097 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf491066-1777-4392-9c3c-a2348b0e7392-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.401200 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.401180 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf491066-1777-4392-9c3c-a2348b0e7392-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.502336 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.502297 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.502336 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.502344 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.502597 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.502362 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf491066-1777-4392-9c3c-a2348b0e7392-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.502597 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.502392 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.502597 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.502420 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.502597 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.502437 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfgdv\" (UniqueName: \"kubernetes.io/projected/cf491066-1777-4392-9c3c-a2348b0e7392-kube-api-access-bfgdv\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.502597 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.502466 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.502597 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.502499 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf491066-1777-4392-9c3c-a2348b0e7392-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.502597 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.502539 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf491066-1777-4392-9c3c-a2348b0e7392-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.502944 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.502742 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-istio-data\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.502944 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.502864 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-workload-certs\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.502944 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.502916 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-credential-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.503189 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.503167 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-workload-socket\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.503300 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.503283 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf491066-1777-4392-9c3c-a2348b0e7392-istiod-ca-cert\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.504707 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.504683 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-istio-envoy\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.504809 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.504758 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf491066-1777-4392-9c3c-a2348b0e7392-istio-podinfo\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.544187 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.544161 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw"] Nov 04 05:47:12.570000 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.569930 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf491066-1777-4392-9c3c-a2348b0e7392-istio-token\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.570178 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.570111 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfgdv\" (UniqueName: \"kubernetes.io/projected/cf491066-1777-4392-9c3c-a2348b0e7392-kube-api-access-bfgdv\") pod \"e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.631751 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.631725 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:12.664735 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.664703 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5"] Nov 04 05:47:12.671320 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.671295 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.705222 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.704331 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.705222 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.704418 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.705222 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.704449 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.705222 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.704519 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.705222 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.704548 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.705222 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.704578 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.705222 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.704609 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5722g\" (UniqueName: \"kubernetes.io/projected/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-kube-api-access-5722g\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.705222 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.704636 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.705222 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.704670 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.739250 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.738670 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5"] Nov 04 05:47:12.805468 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.805434 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.805654 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.805488 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.805654 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.805525 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.805654 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.805557 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5722g\" (UniqueName: \"kubernetes.io/projected/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-kube-api-access-5722g\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.805654 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.805588 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.805654 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.805628 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.805963 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.805698 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.805963 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.805771 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.805963 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.805802 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.805963 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.805828 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-workload-certs\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.805963 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.805895 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-data\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.806302 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.806274 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-credential-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.806462 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.806436 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-workload-socket\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.806590 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.806568 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istiod-ca-cert\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.808126 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.808102 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-envoy\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.808534 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.808511 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-podinfo\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.874784 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.874755 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5722g\" (UniqueName: \"kubernetes.io/projected/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-kube-api-access-5722g\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.883974 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.883947 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-token\") pod \"e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:12.989544 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:12.989514 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:13.094991 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:13.094964 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw"] Nov 04 05:47:13.097256 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:47:13.097230 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf491066_1777_4392_9c3c_a2348b0e7392.slice/crio-d0a0a1d724e7a951eb73bec6996068ceedc65245e19c26053e2ebc04f8433c60 WatchSource:0}: Error finding container d0a0a1d724e7a951eb73bec6996068ceedc65245e19c26053e2ebc04f8433c60: Status 404 returned error can't find the container with id d0a0a1d724e7a951eb73bec6996068ceedc65245e19c26053e2ebc04f8433c60 Nov 04 05:47:13.099464 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:13.099436 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:47:13.099527 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:13.099497 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:47:13.099573 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:13.099526 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:47:13.206591 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:13.206198 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5"] Nov 04 05:47:13.213518 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:13.213483 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw"] Nov 04 05:47:13.214456 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:13.214424 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" podUID="cf234545-c654-43c2-b9ee-a023302087f8" containerName="istio-proxy" containerID="cri-o://d4c7d22652894575687d4e36610c2b574338c31ccd5bc7c23a4c308f4c38ebb7" gracePeriod=30 Nov 04 05:47:13.224142 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:13.221470 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c"] Nov 04 05:47:13.224142 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:13.221703 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" podUID="8aa179d9-60b7-4dec-a891-a379cb56e798" containerName="istio-proxy" containerID="cri-o://712dce42fd00f50a14504a08158db950a4f0aa0d4980c805e9cc6e89b2376675" gracePeriod=30 Nov 04 05:47:13.381415 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:13.381378 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5"] Nov 04 05:47:13.382959 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:47:13.382934 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad3dd9e5_c321_4f2d_a17d_26b22211d1e5.slice/crio-e7d0182f9b7d54bfbf8071aa4eacf8520cb73c21c8a2944113b17eafc06f5a53 WatchSource:0}: Error finding container e7d0182f9b7d54bfbf8071aa4eacf8520cb73c21c8a2944113b17eafc06f5a53: Status 404 returned error can't find the container with id e7d0182f9b7d54bfbf8071aa4eacf8520cb73c21c8a2944113b17eafc06f5a53 Nov 04 05:47:13.385786 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:13.385754 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:47:13.385858 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:13.385834 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:47:13.385901 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:13.385872 5258 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"} Nov 04 05:47:13.681163 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:13.681003 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" event={"ID":"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5","Type":"ContainerStarted","Data":"887fbcd392a59f3c0955ab65753f7788f7dfdef29b68db02c1ea03d149c84712"} Nov 04 05:47:13.681163 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:13.681046 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" podUID="ad3dd9e5-c321-4f2d-a17d-26b22211d1e5" containerName="istio-proxy" containerID="cri-o://887fbcd392a59f3c0955ab65753f7788f7dfdef29b68db02c1ea03d149c84712" gracePeriod=30 Nov 04 05:47:13.681163 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:13.681075 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" event={"ID":"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5","Type":"ContainerStarted","Data":"e7d0182f9b7d54bfbf8071aa4eacf8520cb73c21c8a2944113b17eafc06f5a53"} Nov 04 05:47:13.682773 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:13.682747 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" event={"ID":"cf491066-1777-4392-9c3c-a2348b0e7392","Type":"ContainerStarted","Data":"e1c8b31434fdc2eff96c248ca43f6a7a9a20a4ab829c0d9f72ff5daa193fe136"} Nov 04 05:47:13.682884 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:13.682789 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" event={"ID":"cf491066-1777-4392-9c3c-a2348b0e7392","Type":"ContainerStarted","Data":"d0a0a1d724e7a951eb73bec6996068ceedc65245e19c26053e2ebc04f8433c60"} Nov 04 05:47:13.828523 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:13.828471 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" podStartSLOduration=1.8284554750000002 podStartE2EDuration="1.828455475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:47:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:47:13.826132801 +0000 UTC m=+1167.705525276" watchObservedRunningTime="2026-11-04 05:47:13.828455475 +0000 UTC m=+1167.707847947" Nov 04 05:47:13.962116 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:13.961372 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" podStartSLOduration=1.961355604 podStartE2EDuration="1.961355604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:47:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:47:13.960629036 +0000 UTC m=+1167.840021509" watchObservedRunningTime="2026-11-04 05:47:13.961355604 +0000 UTC m=+1167.840748076" Nov 04 05:47:14.632870 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:14.632832 5258 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:14.659171 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:14.659131 5258 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body= Nov 04 05:47:14.659463 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:14.659189 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" podUID="cf491066-1777-4392-9c3c-a2348b0e7392" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" Nov 04 05:47:15.632278 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:15.632239 5258 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body= Nov 04 05:47:15.632459 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:15.632310 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" podUID="cf491066-1777-4392-9c3c-a2348b0e7392" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" Nov 04 05:47:16.632657 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:16.632623 5258 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body= Nov 04 05:47:16.633104 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:16.632700 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" podUID="cf491066-1777-4392-9c3c-a2348b0e7392" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" Nov 04 05:47:17.622326 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:17.622288 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-6jlll"] Nov 04 05:47:17.632606 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:17.632577 5258 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body= Nov 04 05:47:17.632828 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:17.632806 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" podUID="cf491066-1777-4392-9c3c-a2348b0e7392" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" Nov 04 05:47:18.520962 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.520939 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:47:18.563563 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.563529 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:47:18.599394 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.599366 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf234545-c654-43c2-b9ee-a023302087f8-kube-api-access-rj2c9\" (UniqueName: \"kubernetes.io/projected/cf234545-c654-43c2-b9ee-a023302087f8-kube-api-access-rj2c9\") pod \"cf234545-c654-43c2-b9ee-a023302087f8\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " Nov 04 05:47:18.599536 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.599427 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-istio-data\") pod \"cf234545-c654-43c2-b9ee-a023302087f8\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " Nov 04 05:47:18.599536 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.599496 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cf234545-c654-43c2-b9ee-a023302087f8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf234545-c654-43c2-b9ee-a023302087f8-istio-podinfo\") pod \"cf234545-c654-43c2-b9ee-a023302087f8\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " Nov 04 05:47:18.599536 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.599529 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf234545-c654-43c2-b9ee-a023302087f8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf234545-c654-43c2-b9ee-a023302087f8-istiod-ca-cert\") pod \"cf234545-c654-43c2-b9ee-a023302087f8\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " Nov 04 05:47:18.599636 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.599561 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-istio-envoy\") pod \"cf234545-c654-43c2-b9ee-a023302087f8\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " Nov 04 05:47:18.599636 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.599606 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-workload-certs\") pod \"cf234545-c654-43c2-b9ee-a023302087f8\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " Nov 04 05:47:18.599718 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.599673 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf234545-c654-43c2-b9ee-a023302087f8-istio-token\" (UniqueName: \"kubernetes.io/projected/cf234545-c654-43c2-b9ee-a023302087f8-istio-token\") pod \"cf234545-c654-43c2-b9ee-a023302087f8\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " Nov 04 05:47:18.599771 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.599729 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-credential-socket\") pod \"cf234545-c654-43c2-b9ee-a023302087f8\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " Nov 04 05:47:18.599828 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.599770 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-workload-socket\") pod \"cf234545-c654-43c2-b9ee-a023302087f8\" (UID: \"cf234545-c654-43c2-b9ee-a023302087f8\") " Nov 04 05:47:18.599828 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.599808 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-istio-data" pod "cf234545-c654-43c2-b9ee-a023302087f8" (UID: "cf234545-c654-43c2-b9ee-a023302087f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.600095 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.599909 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-workload-certs" pod "cf234545-c654-43c2-b9ee-a023302087f8" (UID: "cf234545-c654-43c2-b9ee-a023302087f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.600095 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.599962 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf234545-c654-43c2-b9ee-a023302087f8-istiod-ca-cert" pod "cf234545-c654-43c2-b9ee-a023302087f8" (UID: "cf234545-c654-43c2-b9ee-a023302087f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:47:18.600095 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.600017 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-credential-socket" pod "cf234545-c654-43c2-b9ee-a023302087f8" (UID: "cf234545-c654-43c2-b9ee-a023302087f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.600310 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.600179 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-workload-socket" pod "cf234545-c654-43c2-b9ee-a023302087f8" (UID: "cf234545-c654-43c2-b9ee-a023302087f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.600344 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.600319 5258 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-workload-certs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.600344 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.600340 5258 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-credential-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.600429 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.600354 5258 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-workload-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.600429 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.600368 5258 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-istio-data\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.600429 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.600381 5258 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf234545-c654-43c2-b9ee-a023302087f8-istiod-ca-cert\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.601700 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.601677 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-istio-envoy" pod "cf234545-c654-43c2-b9ee-a023302087f8" (UID: "cf234545-c654-43c2-b9ee-a023302087f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.601758 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.601739 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf234545-c654-43c2-b9ee-a023302087f8-kube-api-access-rj2c9" pod "cf234545-c654-43c2-b9ee-a023302087f8" (UID: "cf234545-c654-43c2-b9ee-a023302087f8"). InnerVolumeSpecName "kube-api-access-rj2c9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:18.602230 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.602209 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf234545-c654-43c2-b9ee-a023302087f8-istio-podinfo" pod "cf234545-c654-43c2-b9ee-a023302087f8" (UID: "cf234545-c654-43c2-b9ee-a023302087f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:47:18.602307 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.602226 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf234545-c654-43c2-b9ee-a023302087f8-istio-token" pod "cf234545-c654-43c2-b9ee-a023302087f8" (UID: "cf234545-c654-43c2-b9ee-a023302087f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:18.632278 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.632257 5258 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body= Nov 04 05:47:18.632367 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.632300 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" podUID="cf491066-1777-4392-9c3c-a2348b0e7392" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" Nov 04 05:47:18.701216 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.701191 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-workload-certs\") pod \"8aa179d9-60b7-4dec-a891-a379cb56e798\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " Nov 04 05:47:18.701571 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.701258 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8aa179d9-60b7-4dec-a891-a379cb56e798-kube-api-access-rljdf\" (UniqueName: \"kubernetes.io/projected/8aa179d9-60b7-4dec-a891-a379cb56e798-kube-api-access-rljdf\") pod \"8aa179d9-60b7-4dec-a891-a379cb56e798\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " Nov 04 05:47:18.701571 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.701297 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-workload-socket\") pod \"8aa179d9-60b7-4dec-a891-a379cb56e798\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " Nov 04 05:47:18.701571 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.701320 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-credential-socket\") pod \"8aa179d9-60b7-4dec-a891-a379cb56e798\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " Nov 04 05:47:18.701571 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.701426 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-workload-certs" pod "8aa179d9-60b7-4dec-a891-a379cb56e798" (UID: "8aa179d9-60b7-4dec-a891-a379cb56e798"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.701571 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.701470 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-envoy\") pod \"8aa179d9-60b7-4dec-a891-a379cb56e798\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " Nov 04 05:47:18.701571 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.701520 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8aa179d9-60b7-4dec-a891-a379cb56e798-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8aa179d9-60b7-4dec-a891-a379cb56e798-istiod-ca-cert\") pod \"8aa179d9-60b7-4dec-a891-a379cb56e798\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " Nov 04 05:47:18.701866 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.701585 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-data\") pod \"8aa179d9-60b7-4dec-a891-a379cb56e798\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " Nov 04 05:47:18.701866 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.701595 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-credential-socket" pod "8aa179d9-60b7-4dec-a891-a379cb56e798" (UID: "8aa179d9-60b7-4dec-a891-a379cb56e798"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.701866 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.701604 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-workload-socket" pod "8aa179d9-60b7-4dec-a891-a379cb56e798" (UID: "8aa179d9-60b7-4dec-a891-a379cb56e798"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.701866 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.701635 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-token\" (UniqueName: \"kubernetes.io/projected/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-token\") pod \"8aa179d9-60b7-4dec-a891-a379cb56e798\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " Nov 04 05:47:18.701866 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.701714 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-podinfo\") pod \"8aa179d9-60b7-4dec-a891-a379cb56e798\" (UID: \"8aa179d9-60b7-4dec-a891-a379cb56e798\") " Nov 04 05:47:18.701866 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.701820 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-data" pod "8aa179d9-60b7-4dec-a891-a379cb56e798" (UID: "8aa179d9-60b7-4dec-a891-a379cb56e798"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.702137 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.701883 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8aa179d9-60b7-4dec-a891-a379cb56e798-istiod-ca-cert" pod "8aa179d9-60b7-4dec-a891-a379cb56e798" (UID: "8aa179d9-60b7-4dec-a891-a379cb56e798"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:47:18.702196 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.702149 5258 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf234545-c654-43c2-b9ee-a023302087f8-istio-envoy\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.702196 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.702164 5258 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-workload-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.702196 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.702185 5258 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-credential-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.702353 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.702200 5258 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8aa179d9-60b7-4dec-a891-a379cb56e798-istiod-ca-cert\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.702353 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.702215 5258 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf234545-c654-43c2-b9ee-a023302087f8-istio-token\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.702353 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.702225 5258 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-data\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.702353 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.702233 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rj2c9\" (UniqueName: \"kubernetes.io/projected/cf234545-c654-43c2-b9ee-a023302087f8-kube-api-access-rj2c9\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.702353 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.702245 5258 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-workload-certs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.702353 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.702256 5258 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf234545-c654-43c2-b9ee-a023302087f8-istio-podinfo\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.703672 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.703652 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8aa179d9-60b7-4dec-a891-a379cb56e798-kube-api-access-rljdf" pod "8aa179d9-60b7-4dec-a891-a379cb56e798" (UID: "8aa179d9-60b7-4dec-a891-a379cb56e798"). InnerVolumeSpecName "kube-api-access-rljdf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:18.703798 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.703778 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-podinfo" pod "8aa179d9-60b7-4dec-a891-a379cb56e798" (UID: "8aa179d9-60b7-4dec-a891-a379cb56e798"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:47:18.703949 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.703930 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-token" pod "8aa179d9-60b7-4dec-a891-a379cb56e798" (UID: "8aa179d9-60b7-4dec-a891-a379cb56e798"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:18.704004 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.703937 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-envoy" pod "8aa179d9-60b7-4dec-a891-a379cb56e798" (UID: "8aa179d9-60b7-4dec-a891-a379cb56e798"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:18.715711 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.715686 5258 generic.go:356] "Generic (PLEG): container finished" podID="cf234545-c654-43c2-b9ee-a023302087f8" containerID="d4c7d22652894575687d4e36610c2b574338c31ccd5bc7c23a4c308f4c38ebb7" exitCode=0 Nov 04 05:47:18.716512 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.715758 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" Nov 04 05:47:18.716512 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.715774 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" event={"ID":"cf234545-c654-43c2-b9ee-a023302087f8","Type":"ContainerDied","Data":"d4c7d22652894575687d4e36610c2b574338c31ccd5bc7c23a4c308f4c38ebb7"} Nov 04 05:47:18.716512 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.715840 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw" event={"ID":"cf234545-c654-43c2-b9ee-a023302087f8","Type":"ContainerDied","Data":"73962c7d2ce60854391aa694d1a92dd6ca854716e01d5da7fe8a94ebd653862c"} Nov 04 05:47:18.716512 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.715875 5258 scope.go:122] "RemoveContainer" containerID="d4c7d22652894575687d4e36610c2b574338c31ccd5bc7c23a4c308f4c38ebb7" Nov 04 05:47:18.717350 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.717326 5258 generic.go:356] "Generic (PLEG): container finished" podID="8aa179d9-60b7-4dec-a891-a379cb56e798" containerID="712dce42fd00f50a14504a08158db950a4f0aa0d4980c805e9cc6e89b2376675" exitCode=0 Nov 04 05:47:18.717440 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.717361 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" Nov 04 05:47:18.717440 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.717404 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" event={"ID":"8aa179d9-60b7-4dec-a891-a379cb56e798","Type":"ContainerDied","Data":"712dce42fd00f50a14504a08158db950a4f0aa0d4980c805e9cc6e89b2376675"} Nov 04 05:47:18.717570 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.717441 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c" event={"ID":"8aa179d9-60b7-4dec-a891-a379cb56e798","Type":"ContainerDied","Data":"e1261068cfe8e782aed48aae79d334e72dfbe0c4f65598b15500f3105a2832fc"} Nov 04 05:47:18.794737 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.794712 5258 scope.go:122] "RemoveContainer" containerID="f190f45506b3f8f2b972d0bdfaea00beee8b19df0398064344cda34e657d346a" Nov 04 05:47:18.803475 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.803450 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rljdf\" (UniqueName: \"kubernetes.io/projected/8aa179d9-60b7-4dec-a891-a379cb56e798-kube-api-access-rljdf\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.803545 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.803484 5258 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-envoy\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.803545 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.803499 5258 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-token\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.803545 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.803513 5258 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8aa179d9-60b7-4dec-a891-a379cb56e798-istio-podinfo\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:18.807309 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.807288 5258 scope.go:122] "RemoveContainer" containerID="d4c7d22652894575687d4e36610c2b574338c31ccd5bc7c23a4c308f4c38ebb7" Nov 04 05:47:18.807627 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:47:18.807602 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4c7d22652894575687d4e36610c2b574338c31ccd5bc7c23a4c308f4c38ebb7\": container with ID starting with d4c7d22652894575687d4e36610c2b574338c31ccd5bc7c23a4c308f4c38ebb7 not found: ID does not exist" containerID="d4c7d22652894575687d4e36610c2b574338c31ccd5bc7c23a4c308f4c38ebb7" Nov 04 05:47:18.807715 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.807636 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4c7d22652894575687d4e36610c2b574338c31ccd5bc7c23a4c308f4c38ebb7"} err="failed to get container status \"d4c7d22652894575687d4e36610c2b574338c31ccd5bc7c23a4c308f4c38ebb7\": rpc error: code = NotFound desc = could not find container \"d4c7d22652894575687d4e36610c2b574338c31ccd5bc7c23a4c308f4c38ebb7\": container with ID starting with d4c7d22652894575687d4e36610c2b574338c31ccd5bc7c23a4c308f4c38ebb7 not found: ID does not exist" Nov 04 05:47:18.807715 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.807660 5258 scope.go:122] "RemoveContainer" containerID="f190f45506b3f8f2b972d0bdfaea00beee8b19df0398064344cda34e657d346a" Nov 04 05:47:18.807976 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:47:18.807951 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f190f45506b3f8f2b972d0bdfaea00beee8b19df0398064344cda34e657d346a\": container with ID starting with f190f45506b3f8f2b972d0bdfaea00beee8b19df0398064344cda34e657d346a not found: ID does not exist" containerID="f190f45506b3f8f2b972d0bdfaea00beee8b19df0398064344cda34e657d346a" Nov 04 05:47:18.808054 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.807983 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f190f45506b3f8f2b972d0bdfaea00beee8b19df0398064344cda34e657d346a"} err="failed to get container status \"f190f45506b3f8f2b972d0bdfaea00beee8b19df0398064344cda34e657d346a\": rpc error: code = NotFound desc = could not find container \"f190f45506b3f8f2b972d0bdfaea00beee8b19df0398064344cda34e657d346a\": container with ID starting with f190f45506b3f8f2b972d0bdfaea00beee8b19df0398064344cda34e657d346a not found: ID does not exist" Nov 04 05:47:18.808054 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.808001 5258 scope.go:122] "RemoveContainer" containerID="712dce42fd00f50a14504a08158db950a4f0aa0d4980c805e9cc6e89b2376675" Nov 04 05:47:18.817392 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.817353 5258 scope.go:122] "RemoveContainer" containerID="712dce42fd00f50a14504a08158db950a4f0aa0d4980c805e9cc6e89b2376675" Nov 04 05:47:18.817727 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:47:18.817705 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"712dce42fd00f50a14504a08158db950a4f0aa0d4980c805e9cc6e89b2376675\": container with ID starting with 712dce42fd00f50a14504a08158db950a4f0aa0d4980c805e9cc6e89b2376675 not found: ID does not exist" containerID="712dce42fd00f50a14504a08158db950a4f0aa0d4980c805e9cc6e89b2376675" Nov 04 05:47:18.817821 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.817737 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"712dce42fd00f50a14504a08158db950a4f0aa0d4980c805e9cc6e89b2376675"} err="failed to get container status \"712dce42fd00f50a14504a08158db950a4f0aa0d4980c805e9cc6e89b2376675\": rpc error: code = NotFound desc = could not find container \"712dce42fd00f50a14504a08158db950a4f0aa0d4980c805e9cc6e89b2376675\": container with ID starting with 712dce42fd00f50a14504a08158db950a4f0aa0d4980c805e9cc6e89b2376675 not found: ID does not exist" Nov 04 05:47:18.925822 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.925786 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw"] Nov 04 05:47:18.932471 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.932440 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-gjflw"] Nov 04 05:47:18.981132 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:18.981107 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:19.055415 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.055316 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c"] Nov 04 05:47:19.057515 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.057486 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-smw6c"] Nov 04 05:47:19.106288 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.106258 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-podinfo\") pod \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " Nov 04 05:47:19.106429 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.106297 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-kube-api-access-5722g\" (UniqueName: \"kubernetes.io/projected/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-kube-api-access-5722g\") pod \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " Nov 04 05:47:19.106429 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.106350 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-workload-socket\") pod \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " Nov 04 05:47:19.106429 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.106386 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-token\" (UniqueName: \"kubernetes.io/projected/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-token\") pod \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " Nov 04 05:47:19.106429 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.106409 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-envoy\") pod \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " Nov 04 05:47:19.106636 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.106455 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-data\") pod \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " Nov 04 05:47:19.106636 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.106493 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istiod-ca-cert\") pod \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " Nov 04 05:47:19.106636 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.106520 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-credential-socket\") pod \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " Nov 04 05:47:19.106636 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.106592 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-workload-certs\") pod \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\" (UID: \"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5\") " Nov 04 05:47:19.106857 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.106643 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-workload-socket" pod "ad3dd9e5-c321-4f2d-a17d-26b22211d1e5" (UID: "ad3dd9e5-c321-4f2d-a17d-26b22211d1e5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.106857 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.106725 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-data" pod "ad3dd9e5-c321-4f2d-a17d-26b22211d1e5" (UID: "ad3dd9e5-c321-4f2d-a17d-26b22211d1e5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.106954 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.106863 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-credential-socket" pod "ad3dd9e5-c321-4f2d-a17d-26b22211d1e5" (UID: "ad3dd9e5-c321-4f2d-a17d-26b22211d1e5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.106954 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.106874 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istiod-ca-cert" pod "ad3dd9e5-c321-4f2d-a17d-26b22211d1e5" (UID: "ad3dd9e5-c321-4f2d-a17d-26b22211d1e5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:47:19.107143 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.107112 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-workload-certs" pod "ad3dd9e5-c321-4f2d-a17d-26b22211d1e5" (UID: "ad3dd9e5-c321-4f2d-a17d-26b22211d1e5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.107282 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.107203 5258 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-data\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.107282 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.107224 5258 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istiod-ca-cert\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.107282 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.107238 5258 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-credential-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.107282 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.107254 5258 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-workload-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.108642 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.108614 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-podinfo" pod "ad3dd9e5-c321-4f2d-a17d-26b22211d1e5" (UID: "ad3dd9e5-c321-4f2d-a17d-26b22211d1e5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:47:19.108754 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.108640 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-token" pod "ad3dd9e5-c321-4f2d-a17d-26b22211d1e5" (UID: "ad3dd9e5-c321-4f2d-a17d-26b22211d1e5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:19.108754 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.108658 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-envoy" pod "ad3dd9e5-c321-4f2d-a17d-26b22211d1e5" (UID: "ad3dd9e5-c321-4f2d-a17d-26b22211d1e5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:19.108754 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.108708 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-kube-api-access-5722g" pod "ad3dd9e5-c321-4f2d-a17d-26b22211d1e5" (UID: "ad3dd9e5-c321-4f2d-a17d-26b22211d1e5"). InnerVolumeSpecName "kube-api-access-5722g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:19.208279 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.208234 5258 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-workload-certs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.208279 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.208272 5258 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-podinfo\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.208279 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.208282 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5722g\" (UniqueName: \"kubernetes.io/projected/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-kube-api-access-5722g\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.208516 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.208293 5258 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-token\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.208516 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.208301 5258 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5-istio-envoy\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:19.633169 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.633131 5258 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body= Nov 04 05:47:19.633338 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.633186 5258 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" podUID="cf491066-1777-4392-9c3c-a2348b0e7392" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" Nov 04 05:47:19.721467 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.720827 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25"] Nov 04 05:47:19.721467 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.721111 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" podUID="33978c82-e069-4b5c-bbe9-049f35fb31af" containerName="istio-proxy" containerID="cri-o://068c7091d0485eb0bf3a36ef1bc1d6e482ada7457f0b624fdbc2598717b28cb6" gracePeriod=30 Nov 04 05:47:19.739023 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.738950 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp"] Nov 04 05:47:19.739362 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.739305 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" podUID="0c20521f-5f13-49b4-954d-08b2bc8554d5" containerName="istio-proxy" containerID="cri-o://c30894172b64d0398b1c6f4f465d41e732395708b153386ec578a20264aa433d" gracePeriod=30 Nov 04 05:47:19.753588 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.753554 5258 generic.go:356] "Generic (PLEG): container finished" podID="ad3dd9e5-c321-4f2d-a17d-26b22211d1e5" containerID="887fbcd392a59f3c0955ab65753f7788f7dfdef29b68db02c1ea03d149c84712" exitCode=0 Nov 04 05:47:19.753734 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.753635 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" Nov 04 05:47:19.753734 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.753635 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" event={"ID":"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5","Type":"ContainerDied","Data":"887fbcd392a59f3c0955ab65753f7788f7dfdef29b68db02c1ea03d149c84712"} Nov 04 05:47:19.753734 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.753702 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5" event={"ID":"ad3dd9e5-c321-4f2d-a17d-26b22211d1e5","Type":"ContainerDied","Data":"e7d0182f9b7d54bfbf8071aa4eacf8520cb73c21c8a2944113b17eafc06f5a53"} Nov 04 05:47:19.753734 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.753729 5258 scope.go:122] "RemoveContainer" containerID="887fbcd392a59f3c0955ab65753f7788f7dfdef29b68db02c1ea03d149c84712" Nov 04 05:47:19.758592 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.758572 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw"] Nov 04 05:47:19.758829 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.758809 5258 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" podUID="cf491066-1777-4392-9c3c-a2348b0e7392" containerName="istio-proxy" containerID="cri-o://e1c8b31434fdc2eff96c248ca43f6a7a9a20a4ab829c0d9f72ff5daa193fe136" gracePeriod=30 Nov 04 05:47:19.764503 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.764436 5258 scope.go:122] "RemoveContainer" containerID="887fbcd392a59f3c0955ab65753f7788f7dfdef29b68db02c1ea03d149c84712" Nov 04 05:47:19.764858 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:47:19.764821 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"887fbcd392a59f3c0955ab65753f7788f7dfdef29b68db02c1ea03d149c84712\": container with ID starting with 887fbcd392a59f3c0955ab65753f7788f7dfdef29b68db02c1ea03d149c84712 not found: ID does not exist" containerID="887fbcd392a59f3c0955ab65753f7788f7dfdef29b68db02c1ea03d149c84712" Nov 04 05:47:19.764938 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.764875 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"887fbcd392a59f3c0955ab65753f7788f7dfdef29b68db02c1ea03d149c84712"} err="failed to get container status \"887fbcd392a59f3c0955ab65753f7788f7dfdef29b68db02c1ea03d149c84712\": rpc error: code = NotFound desc = could not find container \"887fbcd392a59f3c0955ab65753f7788f7dfdef29b68db02c1ea03d149c84712\": container with ID starting with 887fbcd392a59f3c0955ab65753f7788f7dfdef29b68db02c1ea03d149c84712 not found: ID does not exist" Nov 04 05:47:19.880003 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.879967 5258 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 05:47:19.880193 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:19.880066 5258 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" podUID="33978c82-e069-4b5c-bbe9-049f35fb31af" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 05:47:20.245856 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:20.245821 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5"] Nov 04 05:47:20.254273 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:20.254239 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-961485-openshift-default-77d8ff86ff-n2mz5"] Nov 04 05:47:20.272098 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:20.272059 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8aa179d9-60b7-4dec-a891-a379cb56e798" path="/var/lib/kubelet/pods/8aa179d9-60b7-4dec-a891-a379cb56e798/volumes" Nov 04 05:47:20.272716 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:20.272696 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ad3dd9e5-c321-4f2d-a17d-26b22211d1e5" path="/var/lib/kubelet/pods/ad3dd9e5-c321-4f2d-a17d-26b22211d1e5/volumes" Nov 04 05:47:20.273335 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:20.273315 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf234545-c654-43c2-b9ee-a023302087f8" path="/var/lib/kubelet/pods/cf234545-c654-43c2-b9ee-a023302087f8/volumes" Nov 04 05:47:22.150950 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:22.150911 5258 patch_prober.go:29] interesting pod/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Nov 04 05:47:22.151502 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:22.150970 5258 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" podUID="0c20521f-5f13-49b4-954d-08b2bc8554d5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 05:47:22.781349 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:47:22.781314 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:47:24.996201 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:24.996147 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:47:25.024166 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.024142 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:47:25.169792 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.169759 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-workload-socket\") pod \"0c20521f-5f13-49b4-954d-08b2bc8554d5\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " Nov 04 05:47:25.169981 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.169807 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-workload-certs\") pod \"0c20521f-5f13-49b4-954d-08b2bc8554d5\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " Nov 04 05:47:25.169981 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.169838 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-credential-socket\") pod \"0c20521f-5f13-49b4-954d-08b2bc8554d5\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " Nov 04 05:47:25.169981 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.169871 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-podinfo\") pod \"33978c82-e069-4b5c-bbe9-049f35fb31af\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " Nov 04 05:47:25.169981 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.169887 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-token\" (UniqueName: \"kubernetes.io/projected/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-token\") pod \"33978c82-e069-4b5c-bbe9-049f35fb31af\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " Nov 04 05:47:25.169981 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.169921 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-workload-socket\") pod \"33978c82-e069-4b5c-bbe9-049f35fb31af\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " Nov 04 05:47:25.169981 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.169953 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-data\") pod \"0c20521f-5f13-49b4-954d-08b2bc8554d5\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " Nov 04 05:47:25.169981 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.169980 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-token\" (UniqueName: \"kubernetes.io/projected/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-token\") pod \"0c20521f-5f13-49b4-954d-08b2bc8554d5\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " Nov 04 05:47:25.170388 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170014 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-workload-certs\") pod \"33978c82-e069-4b5c-bbe9-049f35fb31af\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " Nov 04 05:47:25.170388 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170064 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-envoy\") pod \"0c20521f-5f13-49b4-954d-08b2bc8554d5\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " Nov 04 05:47:25.170388 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170093 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-workload-socket" pod "0c20521f-5f13-49b4-954d-08b2bc8554d5" (UID: "0c20521f-5f13-49b4-954d-08b2bc8554d5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.170388 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170107 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c20521f-5f13-49b4-954d-08b2bc8554d5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c20521f-5f13-49b4-954d-08b2bc8554d5-istiod-ca-cert\") pod \"0c20521f-5f13-49b4-954d-08b2bc8554d5\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " Nov 04 05:47:25.170388 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170194 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-podinfo\") pod \"0c20521f-5f13-49b4-954d-08b2bc8554d5\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " Nov 04 05:47:25.170388 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170220 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-credential-socket" pod "0c20521f-5f13-49b4-954d-08b2bc8554d5" (UID: "0c20521f-5f13-49b4-954d-08b2bc8554d5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.170388 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170258 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-data\") pod \"33978c82-e069-4b5c-bbe9-049f35fb31af\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " Nov 04 05:47:25.170388 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170306 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c20521f-5f13-49b4-954d-08b2bc8554d5-kube-api-access-jjwd6\" (UniqueName: \"kubernetes.io/projected/0c20521f-5f13-49b4-954d-08b2bc8554d5-kube-api-access-jjwd6\") pod \"0c20521f-5f13-49b4-954d-08b2bc8554d5\" (UID: \"0c20521f-5f13-49b4-954d-08b2bc8554d5\") " Nov 04 05:47:25.170388 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170345 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/33978c82-e069-4b5c-bbe9-049f35fb31af-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33978c82-e069-4b5c-bbe9-049f35fb31af-istiod-ca-cert\") pod \"33978c82-e069-4b5c-bbe9-049f35fb31af\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " Nov 04 05:47:25.170388 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170382 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-envoy\") pod \"33978c82-e069-4b5c-bbe9-049f35fb31af\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " Nov 04 05:47:25.170961 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170411 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/33978c82-e069-4b5c-bbe9-049f35fb31af-kube-api-access-n4852\" (UniqueName: \"kubernetes.io/projected/33978c82-e069-4b5c-bbe9-049f35fb31af-kube-api-access-n4852\") pod \"33978c82-e069-4b5c-bbe9-049f35fb31af\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " Nov 04 05:47:25.170961 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170449 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-credential-socket\") pod \"33978c82-e069-4b5c-bbe9-049f35fb31af\" (UID: \"33978c82-e069-4b5c-bbe9-049f35fb31af\") " Nov 04 05:47:25.170961 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170452 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-workload-certs" pod "33978c82-e069-4b5c-bbe9-049f35fb31af" (UID: "33978c82-e069-4b5c-bbe9-049f35fb31af"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.170961 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170457 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-workload-certs" pod "0c20521f-5f13-49b4-954d-08b2bc8554d5" (UID: "0c20521f-5f13-49b4-954d-08b2bc8554d5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.170961 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170493 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c20521f-5f13-49b4-954d-08b2bc8554d5-istiod-ca-cert" pod "0c20521f-5f13-49b4-954d-08b2bc8554d5" (UID: "0c20521f-5f13-49b4-954d-08b2bc8554d5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:47:25.170961 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170715 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-workload-socket" pod "33978c82-e069-4b5c-bbe9-049f35fb31af" (UID: "33978c82-e069-4b5c-bbe9-049f35fb31af"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.170961 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170859 5258 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c20521f-5f13-49b4-954d-08b2bc8554d5-istiod-ca-cert\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.170961 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170882 5258 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-workload-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.170961 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170897 5258 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-workload-certs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.170961 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170913 5258 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-credential-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.170961 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170928 5258 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-workload-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.170961 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170941 5258 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-workload-certs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.171615 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.170993 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-data" pod "33978c82-e069-4b5c-bbe9-049f35fb31af" (UID: "33978c82-e069-4b5c-bbe9-049f35fb31af"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.171615 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.171059 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-data" pod "0c20521f-5f13-49b4-954d-08b2bc8554d5" (UID: "0c20521f-5f13-49b4-954d-08b2bc8554d5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.171615 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.171364 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-credential-socket" pod "33978c82-e069-4b5c-bbe9-049f35fb31af" (UID: "33978c82-e069-4b5c-bbe9-049f35fb31af"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.171615 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.171515 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33978c82-e069-4b5c-bbe9-049f35fb31af-istiod-ca-cert" pod "33978c82-e069-4b5c-bbe9-049f35fb31af" (UID: "33978c82-e069-4b5c-bbe9-049f35fb31af"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:47:25.172609 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.172581 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-podinfo" pod "33978c82-e069-4b5c-bbe9-049f35fb31af" (UID: "33978c82-e069-4b5c-bbe9-049f35fb31af"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:47:25.172920 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.172880 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-token" pod "33978c82-e069-4b5c-bbe9-049f35fb31af" (UID: "33978c82-e069-4b5c-bbe9-049f35fb31af"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:25.172920 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.172905 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-envoy" pod "0c20521f-5f13-49b4-954d-08b2bc8554d5" (UID: "0c20521f-5f13-49b4-954d-08b2bc8554d5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.173095 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.172958 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-podinfo" pod "0c20521f-5f13-49b4-954d-08b2bc8554d5" (UID: "0c20521f-5f13-49b4-954d-08b2bc8554d5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:47:25.173416 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.173399 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c20521f-5f13-49b4-954d-08b2bc8554d5-kube-api-access-jjwd6" pod "0c20521f-5f13-49b4-954d-08b2bc8554d5" (UID: "0c20521f-5f13-49b4-954d-08b2bc8554d5"). InnerVolumeSpecName "kube-api-access-jjwd6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:25.173645 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.173615 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-token" pod "0c20521f-5f13-49b4-954d-08b2bc8554d5" (UID: "0c20521f-5f13-49b4-954d-08b2bc8554d5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:25.173702 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.173643 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-envoy" pod "33978c82-e069-4b5c-bbe9-049f35fb31af" (UID: "33978c82-e069-4b5c-bbe9-049f35fb31af"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.173737 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.173700 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33978c82-e069-4b5c-bbe9-049f35fb31af-kube-api-access-n4852" pod "33978c82-e069-4b5c-bbe9-049f35fb31af" (UID: "33978c82-e069-4b5c-bbe9-049f35fb31af"). InnerVolumeSpecName "kube-api-access-n4852". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:25.272249 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.272171 5258 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-data\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.272249 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.272200 5258 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-token\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.272249 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.272210 5258 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-envoy\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.272249 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.272220 5258 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c20521f-5f13-49b4-954d-08b2bc8554d5-istio-podinfo\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.272249 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.272231 5258 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-data\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.272249 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.272239 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjwd6\" (UniqueName: \"kubernetes.io/projected/0c20521f-5f13-49b4-954d-08b2bc8554d5-kube-api-access-jjwd6\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.272249 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.272248 5258 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/33978c82-e069-4b5c-bbe9-049f35fb31af-istiod-ca-cert\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.272565 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.272257 5258 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-envoy\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.272565 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.272267 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4852\" (UniqueName: \"kubernetes.io/projected/33978c82-e069-4b5c-bbe9-049f35fb31af-kube-api-access-n4852\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.272565 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.272276 5258 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/33978c82-e069-4b5c-bbe9-049f35fb31af-credential-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.272565 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.272286 5258 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-podinfo\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.272565 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.272294 5258 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/33978c82-e069-4b5c-bbe9-049f35fb31af-istio-token\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.797545 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.797516 5258 generic.go:356] "Generic (PLEG): container finished" podID="cf491066-1777-4392-9c3c-a2348b0e7392" containerID="e1c8b31434fdc2eff96c248ca43f6a7a9a20a4ab829c0d9f72ff5daa193fe136" exitCode=0 Nov 04 05:47:25.797675 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.797588 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" event={"ID":"cf491066-1777-4392-9c3c-a2348b0e7392","Type":"ContainerDied","Data":"e1c8b31434fdc2eff96c248ca43f6a7a9a20a4ab829c0d9f72ff5daa193fe136"} Nov 04 05:47:25.797675 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.797643 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" event={"ID":"cf491066-1777-4392-9c3c-a2348b0e7392","Type":"ContainerDied","Data":"d0a0a1d724e7a951eb73bec6996068ceedc65245e19c26053e2ebc04f8433c60"} Nov 04 05:47:25.797675 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.797664 5258 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0a0a1d724e7a951eb73bec6996068ceedc65245e19c26053e2ebc04f8433c60" Nov 04 05:47:25.799078 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.799053 5258 generic.go:356] "Generic (PLEG): container finished" podID="0c20521f-5f13-49b4-954d-08b2bc8554d5" containerID="c30894172b64d0398b1c6f4f465d41e732395708b153386ec578a20264aa433d" exitCode=0 Nov 04 05:47:25.799171 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.799082 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" event={"ID":"0c20521f-5f13-49b4-954d-08b2bc8554d5","Type":"ContainerDied","Data":"c30894172b64d0398b1c6f4f465d41e732395708b153386ec578a20264aa433d"} Nov 04 05:47:25.799171 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.799105 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" Nov 04 05:47:25.799171 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.799131 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp" event={"ID":"0c20521f-5f13-49b4-954d-08b2bc8554d5","Type":"ContainerDied","Data":"5c205838a9a39290f5ec2677a1b670b00d5679616c2fa1dfc1c89d3232397917"} Nov 04 05:47:25.799171 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.799161 5258 scope.go:122] "RemoveContainer" containerID="c30894172b64d0398b1c6f4f465d41e732395708b153386ec578a20264aa433d" Nov 04 05:47:25.800602 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.800579 5258 generic.go:356] "Generic (PLEG): container finished" podID="33978c82-e069-4b5c-bbe9-049f35fb31af" containerID="068c7091d0485eb0bf3a36ef1bc1d6e482ada7457f0b624fdbc2598717b28cb6" exitCode=0 Nov 04 05:47:25.800705 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.800628 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" event={"ID":"33978c82-e069-4b5c-bbe9-049f35fb31af","Type":"ContainerDied","Data":"068c7091d0485eb0bf3a36ef1bc1d6e482ada7457f0b624fdbc2598717b28cb6"} Nov 04 05:47:25.800705 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.800640 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" Nov 04 05:47:25.800705 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.800666 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25" event={"ID":"33978c82-e069-4b5c-bbe9-049f35fb31af","Type":"ContainerDied","Data":"30a379ba89ec571de3c9c273cf3474663a65820fe3f5815476f294cd65601a58"} Nov 04 05:47:25.811353 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.811336 5258 scope.go:122] "RemoveContainer" containerID="c30894172b64d0398b1c6f4f465d41e732395708b153386ec578a20264aa433d" Nov 04 05:47:25.811642 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:47:25.811619 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c30894172b64d0398b1c6f4f465d41e732395708b153386ec578a20264aa433d\": container with ID starting with c30894172b64d0398b1c6f4f465d41e732395708b153386ec578a20264aa433d not found: ID does not exist" containerID="c30894172b64d0398b1c6f4f465d41e732395708b153386ec578a20264aa433d" Nov 04 05:47:25.811748 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.811653 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c30894172b64d0398b1c6f4f465d41e732395708b153386ec578a20264aa433d"} err="failed to get container status \"c30894172b64d0398b1c6f4f465d41e732395708b153386ec578a20264aa433d\": rpc error: code = NotFound desc = could not find container \"c30894172b64d0398b1c6f4f465d41e732395708b153386ec578a20264aa433d\": container with ID starting with c30894172b64d0398b1c6f4f465d41e732395708b153386ec578a20264aa433d not found: ID does not exist" Nov 04 05:47:25.811748 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.811672 5258 scope.go:122] "RemoveContainer" containerID="068c7091d0485eb0bf3a36ef1bc1d6e482ada7457f0b624fdbc2598717b28cb6" Nov 04 05:47:25.816014 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.815995 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:25.820294 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.820270 5258 scope.go:122] "RemoveContainer" containerID="068c7091d0485eb0bf3a36ef1bc1d6e482ada7457f0b624fdbc2598717b28cb6" Nov 04 05:47:25.820564 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:47:25.820545 5258 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"068c7091d0485eb0bf3a36ef1bc1d6e482ada7457f0b624fdbc2598717b28cb6\": container with ID starting with 068c7091d0485eb0bf3a36ef1bc1d6e482ada7457f0b624fdbc2598717b28cb6 not found: ID does not exist" containerID="068c7091d0485eb0bf3a36ef1bc1d6e482ada7457f0b624fdbc2598717b28cb6" Nov 04 05:47:25.820633 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.820570 5258 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"068c7091d0485eb0bf3a36ef1bc1d6e482ada7457f0b624fdbc2598717b28cb6"} err="failed to get container status \"068c7091d0485eb0bf3a36ef1bc1d6e482ada7457f0b624fdbc2598717b28cb6\": rpc error: code = NotFound desc = could not find container \"068c7091d0485eb0bf3a36ef1bc1d6e482ada7457f0b624fdbc2598717b28cb6\": container with ID starting with 068c7091d0485eb0bf3a36ef1bc1d6e482ada7457f0b624fdbc2598717b28cb6 not found: ID does not exist" Nov 04 05:47:25.978177 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.978069 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cf491066-1777-4392-9c3c-a2348b0e7392-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf491066-1777-4392-9c3c-a2348b0e7392-istio-podinfo\") pod \"cf491066-1777-4392-9c3c-a2348b0e7392\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " Nov 04 05:47:25.978177 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.978122 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf491066-1777-4392-9c3c-a2348b0e7392-istio-token\" (UniqueName: \"kubernetes.io/projected/cf491066-1777-4392-9c3c-a2348b0e7392-istio-token\") pod \"cf491066-1777-4392-9c3c-a2348b0e7392\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " Nov 04 05:47:25.978419 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.978190 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-istio-data\") pod \"cf491066-1777-4392-9c3c-a2348b0e7392\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " Nov 04 05:47:25.978419 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.978241 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-workload-socket\") pod \"cf491066-1777-4392-9c3c-a2348b0e7392\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " Nov 04 05:47:25.978419 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.978259 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf491066-1777-4392-9c3c-a2348b0e7392-kube-api-access-bfgdv\" (UniqueName: \"kubernetes.io/projected/cf491066-1777-4392-9c3c-a2348b0e7392-kube-api-access-bfgdv\") pod \"cf491066-1777-4392-9c3c-a2348b0e7392\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " Nov 04 05:47:25.978419 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.978282 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf491066-1777-4392-9c3c-a2348b0e7392-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf491066-1777-4392-9c3c-a2348b0e7392-istiod-ca-cert\") pod \"cf491066-1777-4392-9c3c-a2348b0e7392\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " Nov 04 05:47:25.978419 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.978306 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-workload-certs\") pod \"cf491066-1777-4392-9c3c-a2348b0e7392\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " Nov 04 05:47:25.978419 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.978360 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-credential-socket\") pod \"cf491066-1777-4392-9c3c-a2348b0e7392\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " Nov 04 05:47:25.978419 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.978391 5258 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-istio-envoy\") pod \"cf491066-1777-4392-9c3c-a2348b0e7392\" (UID: \"cf491066-1777-4392-9c3c-a2348b0e7392\") " Nov 04 05:47:25.978784 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.978574 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-workload-socket" pod "cf491066-1777-4392-9c3c-a2348b0e7392" (UID: "cf491066-1777-4392-9c3c-a2348b0e7392"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.978784 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.978673 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf491066-1777-4392-9c3c-a2348b0e7392-istiod-ca-cert" pod "cf491066-1777-4392-9c3c-a2348b0e7392" (UID: "cf491066-1777-4392-9c3c-a2348b0e7392"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 05:47:25.978784 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.978715 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-workload-certs" pod "cf491066-1777-4392-9c3c-a2348b0e7392" (UID: "cf491066-1777-4392-9c3c-a2348b0e7392"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.978937 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.978814 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-istio-data" pod "cf491066-1777-4392-9c3c-a2348b0e7392" (UID: "cf491066-1777-4392-9c3c-a2348b0e7392"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.978937 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.978850 5258 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-workload-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.978937 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.978860 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-credential-socket" pod "cf491066-1777-4392-9c3c-a2348b0e7392" (UID: "cf491066-1777-4392-9c3c-a2348b0e7392"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.978937 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.978885 5258 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf491066-1777-4392-9c3c-a2348b0e7392-istiod-ca-cert\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:25.980474 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.980448 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf491066-1777-4392-9c3c-a2348b0e7392-istio-podinfo" pod "cf491066-1777-4392-9c3c-a2348b0e7392" (UID: "cf491066-1777-4392-9c3c-a2348b0e7392"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 05:47:25.980474 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.980453 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-istio-envoy" pod "cf491066-1777-4392-9c3c-a2348b0e7392" (UID: "cf491066-1777-4392-9c3c-a2348b0e7392"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 05:47:25.980782 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.980759 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf491066-1777-4392-9c3c-a2348b0e7392-istio-token" pod "cf491066-1777-4392-9c3c-a2348b0e7392" (UID: "cf491066-1777-4392-9c3c-a2348b0e7392"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:25.980841 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:25.980779 5258 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf491066-1777-4392-9c3c-a2348b0e7392-kube-api-access-bfgdv" pod "cf491066-1777-4392-9c3c-a2348b0e7392" (UID: "cf491066-1777-4392-9c3c-a2348b0e7392"). InnerVolumeSpecName "kube-api-access-bfgdv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 05:47:26.080190 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:26.080160 5258 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfgdv\" (UniqueName: \"kubernetes.io/projected/cf491066-1777-4392-9c3c-a2348b0e7392-kube-api-access-bfgdv\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:26.080190 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:26.080186 5258 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-workload-certs\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:26.080190 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:26.080196 5258 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-credential-socket\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:26.080190 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:26.080205 5258 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-istio-envoy\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:26.080661 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:26.080213 5258 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf491066-1777-4392-9c3c-a2348b0e7392-istio-podinfo\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:26.080661 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:26.080222 5258 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf491066-1777-4392-9c3c-a2348b0e7392-istio-token\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:26.080661 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:26.080230 5258 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf491066-1777-4392-9c3c-a2348b0e7392-istio-data\") on node \"ip-10-0-130-185.us-west-2.compute.internal\" DevicePath \"\"" Nov 04 05:47:26.116984 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:26.116948 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp"] Nov 04 05:47:26.123990 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:26.123961 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-gd9zp"] Nov 04 05:47:26.246348 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:26.246269 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25"] Nov 04 05:47:26.252260 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:26.252236 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-5nb25"] Nov 04 05:47:26.269825 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:26.269797 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c20521f-5f13-49b4-954d-08b2bc8554d5" path="/var/lib/kubelet/pods/0c20521f-5f13-49b4-954d-08b2bc8554d5/volumes" Nov 04 05:47:26.270315 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:26.270299 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="33978c82-e069-4b5c-bbe9-049f35fb31af" path="/var/lib/kubelet/pods/33978c82-e069-4b5c-bbe9-049f35fb31af/volumes" Nov 04 05:47:26.806919 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:26.806891 5258 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw" Nov 04 05:47:26.991408 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:26.991375 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw"] Nov 04 05:47:27.047491 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:27.047450 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-901a00-openshift-default-57f697764c-j84dw"] Nov 04 05:47:28.269653 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:28.269617 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf491066-1777-4392-9c3c-a2348b0e7392" path="/var/lib/kubelet/pods/cf491066-1777-4392-9c3c-a2348b0e7392/volumes" Nov 04 05:47:34.040753 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:34.040721 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-57c96dfd77-nsdrn_6acc41b2-6de5-4075-bbc2-78975d0b1f6a/manager/0.log" Nov 04 05:47:38.321123 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:38.321090 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-2f464bb2-gpqlr_a0cad87e-348b-45d8-a9b9-cdc83731dbd3/revoke-keys/0.log" Nov 04 05:47:38.553328 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:38.553295 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-4decd865-zfldm_e0bb9631-1874-471a-8c19-6dc5166d75ba/revoke-keys/0.log" Nov 04 05:47:38.785944 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:38.785914 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-57776a9b-9fwsj_6698fb4e-897c-4c3f-adc4-f2887e5ad24d/revoke-keys/0.log" Nov 04 05:47:39.028102 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:39.028059 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-6e443a86-gqh8f_136492af-bd17-4005-96dd-4c57eebb84d9/revoke-keys/0.log" Nov 04 05:47:39.263753 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:39.263715 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-736fd121-nxnn7_4e5142d4-4efa-4dcb-bcd6-add8904b076f/revoke-keys/0.log" Nov 04 05:47:39.739070 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:39.739022 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-7b4ebc2d-w7xb8_7ef5af96-976f-49c3-9d61-ca2cc1369bf2/revoke-keys/0.log" Nov 04 05:47:39.976769 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:39.976742 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-8253ce6b-f78vw_d16c63d4-9853-4510-978f-56d07cd7f192/revoke-keys/0.log" Nov 04 05:47:40.203679 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:40.203653 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-85bc275b-rxc8q_3118c893-72c5-49c2-84c2-bd1b80861767/revoke-keys/0.log" Nov 04 05:47:40.435224 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:40.433341 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-c5ab8c96-lzr2j_6152a395-dec0-41b3-adc6-a5e7294aba66/revoke-keys/0.log" Nov 04 05:47:40.671392 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:40.671355 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-db136c12-pk5gl_1f42dd6a-1809-4cb1-af96-f23fa8cb693c/revoke-keys/0.log" Nov 04 05:47:40.905174 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:40.905139 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-df357346-kn6w8_415eeabb-bacf-453b-801c-cf123ac99615/revoke-keys/0.log" Nov 04 05:47:41.143860 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:41.143831 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-e69da566-4tdk4_8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab/revoke-keys/0.log" Nov 04 05:47:45.351651 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:45.351611 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-6jlll_6a427024-ba5b-4a22-80da-2cb2f8b5bd13/limitador/0.log" Nov 04 05:47:47.819497 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:47.819465 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c6f4696b9-vlb8m_fcb78eed-38d1-4fa3-9697-99f6b0883659/kube-auth-proxy/0.log" Nov 04 05:47:48.059113 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:48.059073 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-8spv4_bf896a57-35de-4283-a01d-ebb28df05d38/istio-proxy/0.log" Nov 04 05:47:49.555061 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:49.554987 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-4rp6q_022ed9ce-af89-4b64-84cd-db4462575f88/payload-processing/0.log" Nov 04 05:47:52.168899 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:52.168854 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd_538f3de1-190f-4977-98e7-aef0bfcaa4b8/main/0.log" Nov 04 05:47:52.642043 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:52.642001 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx_e6b54269-2caf-4d78-82a1-5860b08bc41a/main/0.log" Nov 04 05:47:52.780292 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:47:52.780256 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:47:52.877167 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:47:52.877131 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv_580ba144-f615-4558-ad64-17f0839f01ab/main/0.log" Nov 04 05:48:03.528612 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:48:03.528576 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-57c96dfd77-nsdrn_6acc41b2-6de5-4075-bbc2-78975d0b1f6a/manager/0.log" Nov 04 05:48:07.300574 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:48:07.300545 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-2f464bb2-gpqlr_a0cad87e-348b-45d8-a9b9-cdc83731dbd3/revoke-keys/0.log" Nov 04 05:48:07.555923 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:48:07.555843 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-4decd865-zfldm_e0bb9631-1874-471a-8c19-6dc5166d75ba/revoke-keys/0.log" Nov 04 05:48:07.797780 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:48:07.797749 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-57776a9b-9fwsj_6698fb4e-897c-4c3f-adc4-f2887e5ad24d/revoke-keys/0.log" Nov 04 05:48:08.041404 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:48:08.041379 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-6e443a86-gqh8f_136492af-bd17-4005-96dd-4c57eebb84d9/revoke-keys/0.log" Nov 04 05:48:08.294657 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:48:08.294581 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-736fd121-nxnn7_4e5142d4-4efa-4dcb-bcd6-add8904b076f/revoke-keys/0.log" Nov 04 05:48:08.791491 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:48:08.791451 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-7b4ebc2d-w7xb8_7ef5af96-976f-49c3-9d61-ca2cc1369bf2/revoke-keys/0.log" Nov 04 05:48:09.058090 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:48:09.058001 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-8253ce6b-f78vw_d16c63d4-9853-4510-978f-56d07cd7f192/revoke-keys/0.log" Nov 04 05:48:09.315368 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:48:09.315295 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-85bc275b-rxc8q_3118c893-72c5-49c2-84c2-bd1b80861767/revoke-keys/0.log" Nov 04 05:48:09.562711 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:48:09.562680 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-c5ab8c96-lzr2j_6152a395-dec0-41b3-adc6-a5e7294aba66/revoke-keys/0.log" Nov 04 05:48:09.792248 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:48:09.792224 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-db136c12-pk5gl_1f42dd6a-1809-4cb1-af96-f23fa8cb693c/revoke-keys/0.log" Nov 04 05:48:10.045291 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:48:10.045207 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-df357346-kn6w8_415eeabb-bacf-453b-801c-cf123ac99615/revoke-keys/0.log" Nov 04 05:48:10.285783 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:48:10.285754 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-e69da566-4tdk4_8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab/revoke-keys/0.log" Nov 04 05:48:13.556906 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:48:13.556878 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-6jlll_6a427024-ba5b-4a22-80da-2cb2f8b5bd13/limitador/0.log" Nov 04 05:48:15.317817 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:48:15.317788 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c6f4696b9-vlb8m_fcb78eed-38d1-4fa3-9697-99f6b0883659/kube-auth-proxy/0.log" Nov 04 05:48:15.568926 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:48:15.568829 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-778b7b5c47-8spv4_bf896a57-35de-4283-a01d-ebb28df05d38/istio-proxy/0.log" Nov 04 05:48:17.290329 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:48:17.290298 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-4rp6q_022ed9ce-af89-4b64-84cd-db4462575f88/payload-processing/0.log" Nov 04 05:48:19.301235 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:48:19.301196 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b892zvd_538f3de1-190f-4977-98e7-aef0bfcaa4b8/main/0.log" Nov 04 05:48:19.858291 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:48:19.858262 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-jnwlx_e6b54269-2caf-4d78-82a1-5860b08bc41a/main/0.log" Nov 04 05:48:20.107806 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:48:20.107774 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-h6dbv_580ba144-f615-4558-ad64-17f0839f01ab/main/0.log" Nov 04 05:48:22.780396 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:48:22.780355 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:48:45.141351 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:48:45.141318 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-4rp6q_022ed9ce-af89-4b64-84cd-db4462575f88/payload-processing/0.log" Nov 04 05:48:45.387389 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:48:45.387360 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-4rp6q_022ed9ce-af89-4b64-84cd-db4462575f88/payload-processing/0.log" Nov 04 05:48:52.779747 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:48:52.779705 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:49:19.257237 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.257157 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-4rp6q_022ed9ce-af89-4b64-84cd-db4462575f88/payload-processing/0.log" Nov 04 05:49:19.499064 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.499021 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-4rp6q_022ed9ce-af89-4b64-84cd-db4462575f88/payload-processing/0.log" Nov 04 05:49:19.904682 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.904648 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wtjmg/must-gather-pvd7z"] Nov 04 05:49:19.905072 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.905057 5258 cpu_manager.go:414] "containerMap: removing container" podUID="cf234545-c654-43c2-b9ee-a023302087f8" containerName="istio-proxy" Nov 04 05:49:19.905133 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.905072 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf234545-c654-43c2-b9ee-a023302087f8" containerName="istio-proxy" Nov 04 05:49:19.905133 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.905090 5258 cpu_manager.go:414] "containerMap: removing container" podUID="0c20521f-5f13-49b4-954d-08b2bc8554d5" containerName="istio-proxy" Nov 04 05:49:19.905133 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.905096 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c20521f-5f13-49b4-954d-08b2bc8554d5" containerName="istio-proxy" Nov 04 05:49:19.905133 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.905116 5258 cpu_manager.go:414] "containerMap: removing container" podUID="cf491066-1777-4392-9c3c-a2348b0e7392" containerName="istio-proxy" Nov 04 05:49:19.905133 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.905122 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf491066-1777-4392-9c3c-a2348b0e7392" containerName="istio-proxy" Nov 04 05:49:19.905290 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.905139 5258 cpu_manager.go:414] "containerMap: removing container" podUID="8aa179d9-60b7-4dec-a891-a379cb56e798" containerName="istio-proxy" Nov 04 05:49:19.905290 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.905144 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8aa179d9-60b7-4dec-a891-a379cb56e798" containerName="istio-proxy" Nov 04 05:49:19.905290 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.905153 5258 cpu_manager.go:414] "containerMap: removing container" podUID="ad3dd9e5-c321-4f2d-a17d-26b22211d1e5" containerName="istio-proxy" Nov 04 05:49:19.905290 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.905157 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad3dd9e5-c321-4f2d-a17d-26b22211d1e5" containerName="istio-proxy" Nov 04 05:49:19.905290 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.905168 5258 cpu_manager.go:414] "containerMap: removing container" podUID="33978c82-e069-4b5c-bbe9-049f35fb31af" containerName="istio-proxy" Nov 04 05:49:19.905290 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.905173 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="33978c82-e069-4b5c-bbe9-049f35fb31af" containerName="istio-proxy" Nov 04 05:49:19.905290 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.905180 5258 cpu_manager.go:414] "containerMap: removing container" podUID="cf234545-c654-43c2-b9ee-a023302087f8" containerName="istio-proxy" Nov 04 05:49:19.905290 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.905184 5258 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf234545-c654-43c2-b9ee-a023302087f8" containerName="istio-proxy" Nov 04 05:49:19.905290 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.905240 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="8aa179d9-60b7-4dec-a891-a379cb56e798" containerName="istio-proxy" Nov 04 05:49:19.905290 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.905248 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad3dd9e5-c321-4f2d-a17d-26b22211d1e5" containerName="istio-proxy" Nov 04 05:49:19.905290 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.905255 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="33978c82-e069-4b5c-bbe9-049f35fb31af" containerName="istio-proxy" Nov 04 05:49:19.905290 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.905261 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf234545-c654-43c2-b9ee-a023302087f8" containerName="istio-proxy" Nov 04 05:49:19.905290 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.905267 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf234545-c654-43c2-b9ee-a023302087f8" containerName="istio-proxy" Nov 04 05:49:19.905290 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.905272 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c20521f-5f13-49b4-954d-08b2bc8554d5" containerName="istio-proxy" Nov 04 05:49:19.905290 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.905278 5258 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf491066-1777-4392-9c3c-a2348b0e7392" containerName="istio-proxy" Nov 04 05:49:19.908566 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:19.908550 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wtjmg/must-gather-pvd7z" Nov 04 05:49:20.022337 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:20.022299 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wtjmg\"/\"kube-root-ca.crt\"" Nov 04 05:49:20.022337 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:20.022318 5258 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wtjmg\"/\"openshift-service-ca.crt\"" Nov 04 05:49:20.022337 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:20.022318 5258 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wtjmg\"/\"default-dockercfg-l9jq4\"" Nov 04 05:49:20.024494 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:20.024473 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wtjmg/must-gather-pvd7z"] Nov 04 05:49:20.031711 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:20.031670 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8da3f3d2-936d-495d-966d-b1b67a1b9464-must-gather-output\") pod \"must-gather-pvd7z\" (UID: \"8da3f3d2-936d-495d-966d-b1b67a1b9464\") " pod="openshift-must-gather-wtjmg/must-gather-pvd7z" Nov 04 05:49:20.031711 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:20.031703 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q26s7\" (UniqueName: \"kubernetes.io/projected/8da3f3d2-936d-495d-966d-b1b67a1b9464-kube-api-access-q26s7\") pod \"must-gather-pvd7z\" (UID: \"8da3f3d2-936d-495d-966d-b1b67a1b9464\") " pod="openshift-must-gather-wtjmg/must-gather-pvd7z" Nov 04 05:49:20.080382 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:20.080344 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl"] Nov 04 05:49:20.082265 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:20.082242 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-db136c12-pk5gl"] Nov 04 05:49:20.133259 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:20.133228 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8da3f3d2-936d-495d-966d-b1b67a1b9464-must-gather-output\") pod \"must-gather-pvd7z\" (UID: \"8da3f3d2-936d-495d-966d-b1b67a1b9464\") " pod="openshift-must-gather-wtjmg/must-gather-pvd7z" Nov 04 05:49:20.133259 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:20.133262 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q26s7\" (UniqueName: \"kubernetes.io/projected/8da3f3d2-936d-495d-966d-b1b67a1b9464-kube-api-access-q26s7\") pod \"must-gather-pvd7z\" (UID: \"8da3f3d2-936d-495d-966d-b1b67a1b9464\") " pod="openshift-must-gather-wtjmg/must-gather-pvd7z" Nov 04 05:49:20.133586 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:20.133568 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8da3f3d2-936d-495d-966d-b1b67a1b9464-must-gather-output\") pod \"must-gather-pvd7z\" (UID: \"8da3f3d2-936d-495d-966d-b1b67a1b9464\") " pod="openshift-must-gather-wtjmg/must-gather-pvd7z" Nov 04 05:49:20.199874 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:20.199797 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q26s7\" (UniqueName: \"kubernetes.io/projected/8da3f3d2-936d-495d-966d-b1b67a1b9464-kube-api-access-q26s7\") pod \"must-gather-pvd7z\" (UID: \"8da3f3d2-936d-495d-966d-b1b67a1b9464\") " pod="openshift-must-gather-wtjmg/must-gather-pvd7z" Nov 04 05:49:20.222882 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:20.222853 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wtjmg/must-gather-pvd7z" Nov 04 05:49:20.271249 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:20.271220 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f42dd6a-1809-4cb1-af96-f23fa8cb693c" path="/var/lib/kubelet/pods/1f42dd6a-1809-4cb1-af96-f23fa8cb693c/volumes" Nov 04 05:49:20.609927 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:20.609899 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wtjmg/must-gather-pvd7z"] Nov 04 05:49:20.613287 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:49:20.613253 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8da3f3d2_936d_495d_966d_b1b67a1b9464.slice/crio-af4f1e6a844e28b927b709d2b92eaad085de0b9d8852ffb8b4d1055afdc7c1aa WatchSource:0}: Error finding container af4f1e6a844e28b927b709d2b92eaad085de0b9d8852ffb8b4d1055afdc7c1aa: Status 404 returned error can't find the container with id af4f1e6a844e28b927b709d2b92eaad085de0b9d8852ffb8b4d1055afdc7c1aa Nov 04 05:49:20.615094 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:20.615077 5258 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 05:49:21.050235 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:21.050104 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr"] Nov 04 05:49:21.053943 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:21.053918 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2f464bb2-gpqlr"] Nov 04 05:49:21.377953 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:21.377916 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wtjmg/must-gather-pvd7z" event={"ID":"8da3f3d2-936d-495d-966d-b1b67a1b9464","Type":"ContainerStarted","Data":"af4f1e6a844e28b927b709d2b92eaad085de0b9d8852ffb8b4d1055afdc7c1aa"} Nov 04 05:49:22.049373 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:22.049343 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8"] Nov 04 05:49:22.105851 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:22.105814 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-7b4ebc2d-w7xb8"] Nov 04 05:49:22.270276 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:22.270243 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ef5af96-976f-49c3-9d61-ca2cc1369bf2" path="/var/lib/kubelet/pods/7ef5af96-976f-49c3-9d61-ca2cc1369bf2/volumes" Nov 04 05:49:22.271097 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:22.270679 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a0cad87e-348b-45d8-a9b9-cdc83731dbd3" path="/var/lib/kubelet/pods/a0cad87e-348b-45d8-a9b9-cdc83731dbd3/volumes" Nov 04 05:49:22.384698 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:22.384664 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wtjmg/must-gather-pvd7z" event={"ID":"8da3f3d2-936d-495d-966d-b1b67a1b9464","Type":"ContainerStarted","Data":"caaaa28ec6f6fa32c99ccb70e0b6e0b039a721aac1d4bd88771122bec64b3f6a"} Nov 04 05:49:22.384698 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:22.384705 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wtjmg/must-gather-pvd7z" event={"ID":"8da3f3d2-936d-495d-966d-b1b67a1b9464","Type":"ContainerStarted","Data":"f1677c46b8c8d06b0b7e4c27c39e8577b6874c24ebd5f2ff6ae0f16065b34a7c"} Nov 04 05:49:22.513157 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:22.513097 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-wtjmg/must-gather-pvd7z" podStartSLOduration=2.053274344 podStartE2EDuration="3.51307698s" totalImagesPullingTime="1.459802636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 05:49:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:49:22.511369986 +0000 UTC m=+1296.390762473" watchObservedRunningTime="2026-11-04 05:49:22.51307698 +0000 UTC m=+1296.392469452" Nov 04 05:49:22.779419 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:49:22.779372 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:49:29.078879 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:29.078845 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4pvss_5a710da8-8925-4e16-b43d-fc06a999569b/global-pull-secret-syncer/0.log" Nov 04 05:49:33.056877 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:33.056828 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c5ab8c96-lzr2j"] Nov 04 05:49:33.108952 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:33.108834 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-c5ab8c96-lzr2j"] Nov 04 05:49:33.669350 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:33.669320 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-z2pjx_83db651d-42a9-48ba-a7ef-263e32d237d9/konnectivity-agent/0.log" Nov 04 05:49:34.051679 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:34.051253 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4"] Nov 04 05:49:34.095738 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:34.095512 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-185.us-west-2.compute.internal_4eff7615010b2db89d82737c9bc2bd5b/haproxy/0.log" Nov 04 05:49:34.106939 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:34.106891 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e69da566-4tdk4"] Nov 04 05:49:34.278056 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:34.277863 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6152a395-dec0-41b3-adc6-a5e7294aba66" path="/var/lib/kubelet/pods/6152a395-dec0-41b3-adc6-a5e7294aba66/volumes" Nov 04 05:49:34.282057 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:34.278430 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab" path="/var/lib/kubelet/pods/8d8ef11f-33d3-4fe6-b255-0f015fe2e1ab/volumes" Nov 04 05:49:40.028048 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.027994 5258 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq"] Nov 04 05:49:40.034346 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.034319 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq" Nov 04 05:49:40.044458 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.044428 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81bf88a3-a317-4b37-a259-de8e3a212ff9-lib-modules\") pod \"perf-node-gather-daemonset-hsvdq\" (UID: \"81bf88a3-a317-4b37-a259-de8e3a212ff9\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq" Nov 04 05:49:40.044582 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.044487 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81bf88a3-a317-4b37-a259-de8e3a212ff9-sys\") pod \"perf-node-gather-daemonset-hsvdq\" (UID: \"81bf88a3-a317-4b37-a259-de8e3a212ff9\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq" Nov 04 05:49:40.044625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.044583 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g58l9\" (UniqueName: \"kubernetes.io/projected/81bf88a3-a317-4b37-a259-de8e3a212ff9-kube-api-access-g58l9\") pod \"perf-node-gather-daemonset-hsvdq\" (UID: \"81bf88a3-a317-4b37-a259-de8e3a212ff9\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq" Nov 04 05:49:40.044625 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.044616 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/81bf88a3-a317-4b37-a259-de8e3a212ff9-proc\") pod \"perf-node-gather-daemonset-hsvdq\" (UID: \"81bf88a3-a317-4b37-a259-de8e3a212ff9\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq" Nov 04 05:49:40.044720 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.044631 5258 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/81bf88a3-a317-4b37-a259-de8e3a212ff9-podres\") pod \"perf-node-gather-daemonset-hsvdq\" (UID: \"81bf88a3-a317-4b37-a259-de8e3a212ff9\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq" Nov 04 05:49:40.052731 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.052705 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f"] Nov 04 05:49:40.085000 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.083978 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6e443a86-gqh8f"] Nov 04 05:49:40.108297 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.108262 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq"] Nov 04 05:49:40.146186 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.146149 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81bf88a3-a317-4b37-a259-de8e3a212ff9-lib-modules\") pod \"perf-node-gather-daemonset-hsvdq\" (UID: \"81bf88a3-a317-4b37-a259-de8e3a212ff9\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq" Nov 04 05:49:40.146370 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.146211 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81bf88a3-a317-4b37-a259-de8e3a212ff9-sys\") pod \"perf-node-gather-daemonset-hsvdq\" (UID: \"81bf88a3-a317-4b37-a259-de8e3a212ff9\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq" Nov 04 05:49:40.146370 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.146281 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g58l9\" (UniqueName: \"kubernetes.io/projected/81bf88a3-a317-4b37-a259-de8e3a212ff9-kube-api-access-g58l9\") pod \"perf-node-gather-daemonset-hsvdq\" (UID: \"81bf88a3-a317-4b37-a259-de8e3a212ff9\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq" Nov 04 05:49:40.146370 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.146317 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/81bf88a3-a317-4b37-a259-de8e3a212ff9-proc\") pod \"perf-node-gather-daemonset-hsvdq\" (UID: \"81bf88a3-a317-4b37-a259-de8e3a212ff9\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq" Nov 04 05:49:40.146370 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.146337 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/81bf88a3-a317-4b37-a259-de8e3a212ff9-sys\") pod \"perf-node-gather-daemonset-hsvdq\" (UID: \"81bf88a3-a317-4b37-a259-de8e3a212ff9\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq" Nov 04 05:49:40.146370 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.146315 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/81bf88a3-a317-4b37-a259-de8e3a212ff9-lib-modules\") pod \"perf-node-gather-daemonset-hsvdq\" (UID: \"81bf88a3-a317-4b37-a259-de8e3a212ff9\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq" Nov 04 05:49:40.146370 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.146344 5258 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/81bf88a3-a317-4b37-a259-de8e3a212ff9-podres\") pod \"perf-node-gather-daemonset-hsvdq\" (UID: \"81bf88a3-a317-4b37-a259-de8e3a212ff9\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq" Nov 04 05:49:40.146677 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.146419 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/81bf88a3-a317-4b37-a259-de8e3a212ff9-proc\") pod \"perf-node-gather-daemonset-hsvdq\" (UID: \"81bf88a3-a317-4b37-a259-de8e3a212ff9\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq" Nov 04 05:49:40.146677 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.146474 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/81bf88a3-a317-4b37-a259-de8e3a212ff9-podres\") pod \"perf-node-gather-daemonset-hsvdq\" (UID: \"81bf88a3-a317-4b37-a259-de8e3a212ff9\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq" Nov 04 05:49:40.213211 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.213177 5258 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g58l9\" (UniqueName: \"kubernetes.io/projected/81bf88a3-a317-4b37-a259-de8e3a212ff9-kube-api-access-g58l9\") pod \"perf-node-gather-daemonset-hsvdq\" (UID: \"81bf88a3-a317-4b37-a259-de8e3a212ff9\") " pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq" Nov 04 05:49:40.270011 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.269969 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="136492af-bd17-4005-96dd-4c57eebb84d9" path="/var/lib/kubelet/pods/136492af-bd17-4005-96dd-4c57eebb84d9/volumes" Nov 04 05:49:40.348366 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.348281 5258 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq" Nov 04 05:49:40.744757 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:40.744720 5258 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq"] Nov 04 05:49:40.748481 ip-10-0-130-185 kubenswrapper[5258]: W1104 05:49:40.748444 5258 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod81bf88a3_a317_4b37_a259_de8e3a212ff9.slice/crio-236de10face4b47964bb8d20b3d21bbd3414152aa4be300c59a659283db0655d WatchSource:0}: Error finding container 236de10face4b47964bb8d20b3d21bbd3414152aa4be300c59a659283db0655d: Status 404 returned error can't find the container with id 236de10face4b47964bb8d20b3d21bbd3414152aa4be300c59a659283db0655d Nov 04 05:49:41.570747 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:41.570692 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq" event={"ID":"81bf88a3-a317-4b37-a259-de8e3a212ff9","Type":"ContainerStarted","Data":"38d05aa3974cba786b1f6cdeeefa6bc2400149c0b2090d3c78c6d0dc1f46d1ea"} Nov 04 05:49:41.571304 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:41.570838 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq" Nov 04 05:49:41.571304 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:41.570866 5258 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq" event={"ID":"81bf88a3-a317-4b37-a259-de8e3a212ff9","Type":"ContainerStarted","Data":"236de10face4b47964bb8d20b3d21bbd3414152aa4be300c59a659283db0655d"} Nov 04 05:49:41.699666 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:41.699616 5258 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq" podStartSLOduration=2.6996034079999998 podStartE2EDuration="2.699603408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 05:49:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 05:49:41.699499645 +0000 UTC m=+1315.578892121" watchObservedRunningTime="2026-11-04 05:49:41.699603408 +0000 UTC m=+1315.578995881" Nov 04 05:49:43.692155 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:43.692120 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-6jlll_6a427024-ba5b-4a22-80da-2cb2f8b5bd13/limitador/0.log" Nov 04 05:49:44.054426 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:44.054333 5258 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj"] Nov 04 05:49:44.110227 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:44.110192 5258 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-57776a9b-9fwsj"] Nov 04 05:49:44.270360 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:44.270325 5258 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6698fb4e-897c-4c3f-adc4-f2887e5ad24d" path="/var/lib/kubelet/pods/6698fb4e-897c-4c3f-adc4-f2887e5ad24d/volumes" Nov 04 05:49:47.588484 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:47.588449 5258 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wtjmg/perf-node-gather-daemonset-hsvdq" Nov 04 05:49:50.622851 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:50.622818 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-p2pjj_9909a4b9-77f0-4be2-9374-26a602a72987/kube-state-metrics/0.log" Nov 04 05:49:51.082813 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:51.082736 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-p2pjj_9909a4b9-77f0-4be2-9374-26a602a72987/kube-rbac-proxy-main/0.log" Nov 04 05:49:51.554897 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:51.554867 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-p2pjj_9909a4b9-77f0-4be2-9374-26a602a72987/kube-rbac-proxy-self/0.log" Nov 04 05:49:52.022136 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:52.022107 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-78bc77fdbf-nrvds_5cf82613-6edf-4e3d-b6cc-b26d3d2da6a2/metrics-server/0.log" Nov 04 05:49:52.488568 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:52.488540 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-dcqtd_07d855d9-bce9-42be-9acf-df2386b300ab/monitoring-plugin/0.log" Nov 04 05:49:52.780402 ip-10-0-130-185 kubenswrapper[5258]: E1104 05:49:52.780318 5258 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-qrxks" containerName="extractor" Nov 04 05:49:52.964550 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:52.964516 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-69gbq_7810845b-042a-4fe6-bc58-cf7285f7e396/node-exporter/0.log" Nov 04 05:49:53.430065 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:53.429265 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-69gbq_7810845b-042a-4fe6-bc58-cf7285f7e396/kube-rbac-proxy/0.log" Nov 04 05:49:53.900402 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:53.900376 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-69gbq_7810845b-042a-4fe6-bc58-cf7285f7e396/init-textfile/0.log" Nov 04 05:49:57.171599 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:57.171565 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-pbhxj_4dcc3e21-060d-4d88-a034-025ee61af3ff/kube-rbac-proxy-main/0.log" Nov 04 05:49:57.647049 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:57.647001 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-pbhxj_4dcc3e21-060d-4d88-a034-025ee61af3ff/kube-rbac-proxy-self/0.log" Nov 04 05:49:58.119837 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:58.119806 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-pbhxj_4dcc3e21-060d-4d88-a034-025ee61af3ff/openshift-state-metrics/0.log" Nov 04 05:49:58.597513 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:58.597430 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_add07311-410f-451b-9010-f8f88a49d33a/prometheus/0.log" Nov 04 05:49:59.060588 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:59.060553 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_add07311-410f-451b-9010-f8f88a49d33a/config-reloader/0.log" Nov 04 05:49:59.525187 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:59.525147 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_add07311-410f-451b-9010-f8f88a49d33a/thanos-sidecar/0.log" Nov 04 05:49:59.989705 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:49:59.989672 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_add07311-410f-451b-9010-f8f88a49d33a/kube-rbac-proxy-web/0.log" Nov 04 05:50:00.456146 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:50:00.456113 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_add07311-410f-451b-9010-f8f88a49d33a/kube-rbac-proxy/0.log" Nov 04 05:50:00.920967 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:50:00.920937 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_add07311-410f-451b-9010-f8f88a49d33a/kube-rbac-proxy-thanos/0.log" Nov 04 05:50:01.384850 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:50:01.384822 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_add07311-410f-451b-9010-f8f88a49d33a/init-config-reloader/0.log" Nov 04 05:50:02.779489 ip-10-0-130-185 kubenswrapper[5258]: I1104 05:50:02.779436 5258 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-pnm9q_d7741ae3-0846-4449-a738-45251ad287df/prometheus-operator-admission-webhook/0.log"